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Abstract. Model checking linear-time properties expressed in first-order logic has non- 
elementary complexity, and thus various restricted logical languages are employed. In this 
paper we consider two such restricted specification logics, linear temporal logic (LTL) and 
two-variable first-order logic (FO^). LTL is more expressive but FO^ can be more succinct, 
and hence it is not clear which should be easier to verify. We take a comprehensive look 
at the issue, giving a comparison of verification problems for FO^, LTL, and various sub- 
logics thereof across a wide range of models. In particular, we look at unary temporal logic 
(UTL), a subset of LTL that is expressively equivalent to FO^; we also consider the stutter- 
free fragment of FO^, obtained by omitting the successor relation, and the expressively 
equivalent fragment of UTL, obtained by omitting the next and previous connectives. 

We give three logic-to-automata translations which can be used to give upper bounds 
for FO^ and UTL and various sub-logics. We apply these to get new bounds for both non- 
deterministic systems (hierarchical and recursive state machines, games) and for proba- 
bilistic systems (Markov chains, recursive Markov chains, and Markov decision processes). 
We couple these with matching lower-bound arguments. 

Next, we look at combining FO^ verification techniques with those for LTL. We present 
here a language that subsumes both FO^ and LTL, and inherits the model checking prop- 
erties of both languages. Our results give both a unified approach to understanding the 
behaviour of FO'^ and LTL, along with a nearly comprehensive picture of the complexity 
of verification for these logics and their sublogics. 



The complexity of verification problems clearly depends on the specification language for 
describing properties. Arguably the most important such language is Linear Temporal Logic 
(LTL). LTL has a simple syntax, one can verify LTL properties over Kripke structures in 
polynomial space, and one can check satisfiability also in polynomial space. Moreover, 
Kamp |Kam68j has shown that LTL has the same expressiveness as first-order logic over 
words. For example, the first-order property "after we are born, we live until we die": 

Vx {horn[x) 3y > x die{y) A Vz (x < z < y — )• live{z))) 

2012 ACM CCS: [Theory of computation]: Logic — Verification by model checking; Formal languages 
and automata theory; Models of computation — Abstract machines. 
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is expressed in LTL by the formula □(6orn — )• live U die). 

In contrast with LTL, model checking first-order queries has non-elementary complex- 
ity |Sto74j — thus LTL could be thought of as a tractable syntactic fragment of FO. Another 
approach to obtaining tractability within first-order logic is by maintaining first-order syn- 
tax, but restricting to two- variable formulas. The resulting specification language FO^ has 
also been shown to have dramatically lower complexity than full first-order logic. In particu- 
lar, Etessami, Vardi and Wilke |EVW02] showed that satisfiabihty for FO^ is NEXPTIME- 
complete and that FO^ is strictly less expressive than FO (and thus less expressive than 
LTL also). Indeed, |EVWn2j shows that FO^ has the same expressive power as Unary Tem- 
poral Logic (UTL): the fragment of LTL with only the unary operators "previous", "next", 
"sometime in the past", "sometime in the future". Consider the example above. We have 
shown that it can be expressed in LTL, but it is easy to show that it cannot be expressed 
in UTL, and therefore cannot be expressed in FO^. 

Although FO^ is less expressive than LTL, there are some properties that are signifi- 
cantly easier to express in FO^ than in LTL. Consider the property that two n-bit identifiers 
agree: 

3x3y{x<yA /\ hi{x) ^ h,{y)) . 

l<i<n 

It is easy to show that there is an exponential blow-up in transforming the above FO^ 
formula into an equivalent LTL formula. We thus have three languages UTL, LTL and FO^, 
with UTL and FO^ equally expressive, LTL more expressive, and with FO^ incomparable 
in succinctness with LTL. 

Are verification tasks easier to perform in LTL, or in FO^? This is the main question 
we address in this paper. There are well-known examples of problems that are easier in 
LTL than in FO^: in particular satisfiability, which is PSPACE-complete for LTL and 
NEXPTIME-complete for FO^ jEVW02j . We will show that there are also tasks where 
FO^ is more tractable than LTL. 

Our main contribution is a uniform approach to the verification of FO^ via automata. 
We show that translations to the appropriate automata can give optimal bounds for veri- 
fication of FO^ on both non-deterministic and probabilistic structures. We also show that 
such translations allow us to understand the verification of the fragment of FO^ formed by 
removing the successor relation from the signature, denoted F0^[<]. It turns out, some- 
what surprisingly, that for this fragment we can get the same complexity upper bounds 
for verification as for the simplest temporal logic — TL[0,<$>]. For our translations from 
F0^[<] to automata, we make use of a key result from Weis [Weill j . showing that models 
of F0^[<] formulas realise only a polynomial number of types. We extend this "few types" 
result from finite to infinite words and use it to characterise the structure of automata for 
F02[<]. 

The outcome of our translations is a comprehensive analysis of the complexity of FO^ 
and UTL verification problems, together with those for the respective stutter- free fragments 
F0^[<] and TL[0,<$>]. We begin with model checking problems for Kripke structures and 
for recursive state machines (RSMs), which we compare to known results for LTL on these 
models. We then turn to two-player games, considering the complexity of the problem of 
determining which player has a strategy to ensure that a given formula is satisfied. We 
then move from non-deterministic systems to probabilistic systems. We start with Markov 
chains and recursive Markov chains, the analogs of Kripke structures and RSMs in the 
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probabilistic case. Finally we consider one-player stochastic games, looking at the question 
of whether the player can devise a strategy that is winning with a given probability. 

Towards the end of the paper, we consider extensions of FO^, and in particular how FO'^ 
verification techniques can be combined with those for Linear Temporal Logic (LTL). We 
present here a language that we denote FO^[LTL], subsuming both FO^ and LTL. We show 
that the complexity of verification problems for FO^[LTL] can be attacked by our automata- 
theoretic methods, and indeed reduces to verification of FO^ and LTL individually. As 
a result we show that the worst-case complexity of probabilistic verification, as well as 
non-deterministic verification, for FO^[LTL] is (roughly speaking) the maximum of the 
complexity for FO'^ and LTL. 

This paper expands on results presented in two conference papers, jBLWll( IBLW12] . 

Organization: Section [2] contains preliminaries, while Section |3] gives fundamental 
results on the model theory of FO^ and its relation to UTL that will be used in the remainder 
of the paper. Section [4] presents the logic-to-automata translations used in our upper 
bounds. The first is a translation of a given UTL formula to a large disjoint union of Biichi 
automata with certain structural restrictions. This can also be used to give a translation 
from a given FO^ formula to an (still larger) union of Biichi automata. The second does 
something similar for F0^[<] formulas. The last translation maps F0^[<] and FO^ formulas 
to deterministic parity automata, which is useful for certain problems involving games. 

Section |6] gives upper and lower bounds for non-deterministic systems, while Section[7]is 
concerned with probabilistic systems. In Section [s] we consider model checking of FO^[LTL], 
which subsumes both FO^ and LTL, and finally in Section [o] we consider the impact of 
extending all the previous logics with let definitions. 

2. Logic, Automata and Complexity Classes 

We consider a first-order signature with set of unary predicates V = {Pi, . . . , Pm} and 
binary predicates < (less than) and sue (successor). Fixing two distinct variables x and y, 
we denote by FO^ the set of first-order formulas over the above signature involving only the 
variables x and y. We denote by F0^[<] the sublogic in which the binary predicate sue is 
not used. We write (p{x) for a formula in which only the variable x occurs free. 

In this paper we are interested in interpretations of FO^ on infinite words. An w-word 
u = uqui . . . over the powerset alphabet S = 2^ represents a first-order structure extending 
(N, <, sue), in which predicate Pi is interpreted by the set {n G N : -Pj G Un\ and the binary 
predicates < and sue have the obvious interpretations. 

We also consider Linear Temporal Logic LTL on w-words. The formulas of LTL are 
built from atomic propositions using Boolean connectives and the temporal operators O 
[next), Q (previously), O (eventually), <$> (sometime in the past), U (until), and S (since). 
Formally, LTL is defined by the following grammar: 

Lp ::= Pi \ Lp f\ Lp \ —iLp I LplA tp I if S ip I Oyj I (p I O ip | Q cp , 

where Pq,Pi, . . . are propositional variables. Unary temporal logic (UTL) denotes the subset 
without U and S, while TL[0>^] denotes the stutter-free subset of UTL without O and 
0. We use \Z\ip as an abbreviation for -lO""/?- 

Let (u, i) be the suffix UiUi^i ... of w-word u. We define the semantics of LTL inductively 
on the structure of the formulas as follows: 
(1) (u, i) 1= Pk iff atomic prop. P^ holds at position i of u 
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It is well known that over w-words LTL has the same expressiveness as first-order logic, 
and UTL has the same expressiveness as FO^. Moreover, while FO^ is less expressive than 
LTL, it can be exponentially more succinct [EVW02] - for concrete examples of these facts, 
see the introduction. 

FO, LTL. F02[LTL] 
i 

FO^. UTL 
1 

F02[<], TL[0,^] 

Figure 1: Expressiveness Diagram 

We can combine the succinctness of FO^ and the expressiveness of LTL by extending 
the former with the temporal operators lA and S (applied to formulas with at most one free 
variable). We call the resulting logic FO^[LTL]. The syntax of FO^[LTL] divides formulas 
into two syntactic classes: temporal formulas and first-order formulas. Temporal formulas 
are given by the grammar 

99 ::= Pj I 99 A 99 I -199 I 99 99 I 99 5 99 I V ) 

where Pi is an atomic proposition and ifj \s a first-order formula with one free variable. 
First-order formulas are given by the grammar 

il) ::= ip{x) I x<y I suc(x, y) \ if) /xijj \ -■■i/' | BxV', 

where 99 is a temporal formula. Here the first-order formula 99(x) asserts that the temporal 
formula 99 holds at position x. The temporal operators O, 0, O and <$> can all be introduced 
as derived operators. An example of FO^[LTL] formula is: 

boU {3y{y <xA /\ bi{x) ^ hi{y))) . 

l<i<n 

The relative expressiveness of the logics defined thus far is illustrated in Figure [T} 

Finally, we consider an extension of FO^[LTL] with let definitions. We inductively define 
the formulas and the unary predicate subformulas that occur free in such a formula. The 
atomic formulas of FO^[LTL]|_ot are as in F02[LTL], with the formula P{ x) occurring freely 
in itself. The constructors include all those of FO^[LTL], with the set of free subformula 
occurrences being preserved by all of these constructors. 
There is one new formula constructor of the form: 

99 ::= Let Pi{x) be 991(2;) in 992 
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where Pi is a unary predicate, ^i{x) is an FO^[LTL]Let formula in which x is the only 
free variable and no occurrence of predicate Pi is free, and ip2 is an arbitrary FO'^ [LTL] Let 
formula. A subformula Pj{z) occurs freely in '^{x) iff it occurs freely in <^i(x) or it occurs 
freely in (p2 and the predicate is not Pi. 

The semantics of FO^[LTL]|_et is defined via a translation function T to FO^[LTL], with 
the only non-trivial rule being: 

r(Let Pi{x) be in ip2) ::= 

T{^2[Piix) ^ n^i), P^{y) ^ Ti^i)[x ^ y]]) 
where T{(pi)[x i— t- y] denotes the formula obtained by substituting variable y for all free 
occurrences of x in T(ipi), and T{ip2[Pi{x) i— >• T{ipi), Pi(y) i— )• T{ipi)[x i— >• y]]) denotes 
substitution of any free occurrence of the form Pi{x) in T{(pi) and every occurrence of Pi{y) 
by T{ipi)[x I— y]. We let UTLLet be the extension of UTL by the operator above, and 
similarly define TL[0,0]Let, F0^[<] 

Let) Stc. 

For if a temporal logic formula or an FO^ formula with one free variable, we denote by 
L{(p) the set {w £ S"^ : {w,0) |= 93} of infinite words that satisfy ip at the initial position. 
The quantifier depth of an FO^ formula (p is denoted qdp{ip) and the operator depth of a 
UTL formula if is denoted odp{ip). In either case the length of the formula is denoted \(p\. 

The notion of a subformula of an FO^[LTL] formula is defined as usual. For an 
FO^[LTL]Let formula ip, let suh{ip) denote the set of subformulas of the equivalent FO^[LTL] 
formula T{(p), where T is the translation function defined above. 

Lemma 2.1. Given an LTLLct formula (p, |sub(99)| is linear in \ip\. 

Proof. Notice that li (p = Let Pi{x) be (pi{x) in ^p2{x), then |sub(((?)| < |sub((/?i)| + |sub((/92)|. 
Then by structural induction it holds that for a LTLLot-formula <p>, sub(99) has size at most 

wv □ 

Biichi Automata. Our results will be obtained via transforming formulas to automata 
that accept w- words. We will be most concerned with generalised Biichi automata (GBA). 
A GBA ^ is a tuple (S, 5*, Sq, A, J^) with alphabet S, set of states S, set of initial states 
5*0 ^ -S*, transition function A and set of sets of final states J-. The accepting condition is 
that for each F £ there is a state s £ F which is visited infinitely often. We can have 
labels either on states or on transitions, but both models are equivalent. For more details, 
see |VW86j . We will consider two important classes of Biichi automata: the automaton 
A is said to be deterministic in the limit if all states reachable from accepting states are 
deterministic; A is unambiguous if for each state s each word is accepted along at most one 
run that starts at s. 

Deterministic Parity Automata. For some model checking problems, we will need 
to work with deterministic automata. In particular, we will use deterministic parity au- 
tomata. A deterministic parity automaton ^ is a tuple (T,, S, sq, A, Pr) with alphabet S, 
set of states S, an initial state sq £ S, transition function A and a priority function Pr 
mapping each state to a natural number. The transition function A maps each state and 
symbol of the alphabet exactly to one new state. A run of such an automaton on input uj- 
word induces an infinite sequence of priorities. The acceptance condition is that the highest 
infinitely often occurring priority in this sequence is even. 

Complexity Classes. Our complexity bounds involve counting classes. #P is the 
class of functions / for which there is a non-deterministic polynomial-time Turing Ma- 
chine T such that f{x) is the number of accepting computation paths of T on input x. A 
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complete problem for ^^P is #SAT, the problem of counting the number of satisfying as- 
signments of a given boolean formula. We will be considering computations of probabilities, 
not integers, so our problems will technically not be in ^^P; but some of them will have 
representations computable in the related class FP"^^, and will be #P-hard. For brevity, 
we will sometimes abuse notation by saying that such probability computation problems 
are #P-complete. The class of functions #EXP is defined analogously to #P, except with 
T a non-deterministic exponential-time machine. We will deal with a decision version of 
T^EXP, PEXP, the set of problems solvable by nondeterministic Turing machine in expo- 
nential time, where the acceptance condition is that more than a half of computation paths 
accept f BFT98j . 

Notation: In our complexity bounds, we will often write poly to denote a fixed but 
arbitrary polynomial. 



3. FO^ MODEL THEORY AND SUCCINCTNESS 

We now discuss the model theory of FO^, summarizing and slightly extending the material 
presented in Etessami, Vardi, and Wilke jEVW02j and in Weis and Immerman |WI09] . 

Recall that we will consider strings over alphabet S = 2^, where V is the set of unary 
predicates appearing in the input F0^[<] formula. We start by recalling the small- model 
property of FO^ that underlies the NEXPTIME satisfiability result of Etessami, Vardi, and 
Wilke |EVWn2j . it is also imphcit in Theorem 6.2 of |Win9] . 

The domain of a word n G S*US'^ is the set dom{u) = {i £ N : < i < |n|} of positions 
in u. The range of u is the set ran(u) = {ui : i G dom(n)} of letters occurring in u. Write 
also inf (u) for the set of letters that occur infinitely often in u. 

Given a finite or infinite word n G S* U S^, a position i € dom(u), and /c € N, we define 
the k-type of u at position i to be the set of F0^[<] formulas 

Tk{u^i) = {'^{x) : qdp((/?) = k and (n, i) |= (/?} . 

Given n, f € S* U and positions i G dom(n) and j G dom(t>), write (u, ~fc {v^j) if 
and only if rfc(M, i) = Tfc(f , j). Furthermore, we write u v for two strings u, w G S* U S'^ if 
for all FO^[<]-formulas ^{x) of quantifier depth at most k we have (n, 0) |= if iff (f , 0) |= (p. 

The small model property of |iEVW02 ] can then be stated as follows: 

Proposition 3.1 ( |EVWn2j ). Let E = 2^. Then (i) For any string n G S* and positive 
integer k there exists f; G S* such that u ~fc v and \ v\ G 2^^^'^^^'^ ; (ii) for any infinite string 
u G S'^ and positive integer k there are finite strings v and w, with \v\, \vu\ G 2'^(l^l^), such 
that u ~fc vw^ . 



For completeness, we give a constructive proof of Propos ition 3.1 which will be used in 
one of our translations of FO^ to automata. This is Lemma l3.9l at the end of this section. 
For this it is convenient to use the following inductive characterisation of which is 
proven in |EVW02] by a straightforward induction: 

Proposition 3.2 ( |EVW02j ). Let n, G S* U S'^. Then Tk{u,i) = Tk{v,j) if and only if 
(i) Ui = Vj, (ii) {Tfc_i(u,i') :i'<i} = {Tk-i{v,j') : / < j}, and (Hi) {Tk-i{u,i') : i' > i} = 
{rk-i{v,j') ■■)'>)}■ 

The next proposition states that we can collapse any two positions in a string that have 
the same /c-type without affecting the fe-type of the string. 
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Proposition 3.3 ([EVW02]). Let n E S* U S"^ and let i < j be such that {u,i) {u,j)- 
Writing u = ui . . . Uju' , we have u ui . . . Uiu' . 

From these two propositions it follows that every finite string is equivalent under to 
a string of length exponential in k and \V\. 

Proposition 3.4. Given a nonnegative integer k, for all strings u £ T,* there exists a string 
V £ T,* such that u v and \v\ is hounded by 2'^^^'^^''\ 

Proof. We prove by induction on k that the set {t^^u, i) : i £ dom(u)} of fc-types occurring 
along u has size at most |S|(2|5]| + 2)^. 
The base case A; = is clear. 

For the induction step, assume that the number of {k — l)-types occurring along u is 
at most |S|(2|S| + 2)^^"^. Define a boundary point in u to be the position of the first or 
last occurrence of a given {k — l)-type. Then there are at most 2|S|(2|S| + 2)'^"^ boundary 



points. But by Proposition 3.2 the A:-type at a given position i in u is determined by Ui, the 
set of boundary points strictly less than i, and the set of boundary points strictly greater 
than i. Thus the number of A:-types along u is at most 

(|S| + l)2|S|(2|i;| + 2)'=-i = |S|(2|S| + 2)'' . (3.1) 



By Proposition 3.3 given any string v in which there are two distinct positions with 



the same fc-type there exists a shorter string w with v w. From the bound (3.1) on the 
number of boundary points, we conclude that there exists a string v such that u v and 
|?;| < |S|(2|S| +2)^ e 20(l^|fc). □ 

The relation is also easy to compute: 

Proposition 3.5. Given u,v G T,* of length at most h we can compute whether u v in 
time at most 

Proof. For m = 0,1, ... ,k we successively pass along u labelling each position i with its 
m-type Tm{u,i). Each rank m requires two passes: we pass leftward through u computing 
the set of {m — l)-types to the left of each position, then we pass rightward computing the 
set of (m — l)-types to the right of each position. This requires 2k passes, with each pass 
taking time linear in h and at most quadratic in the number of fe-types that occur along 
u. The bound now follows using the estimate of the number of types given in Proposition 
[33 □ 



Combining Propositions 3.4 and 3.5 we get: 



Corollary 3.6. Given k there exists a set Repfc(S) C S* of representative strings such 
that each v G Repk{^) has \v\ < |S|(2|S| + 2)'' and for each string n G S* there exists a 
unique v € Rep^(S) such that u v. Moreover Rep;j.(S) can be computed from k in time 

The following result is classical, and can be proven using games. 

Proposition 3.7. Given u,v £ T,* and u',v' £ S'^, for all k if u v and u' v' then 
uu' ~jt vv' . 

From the above we infer that the equivalence class of an infinite string under is 
determined by a prefix of the string and the set of letters appearing infinitely often within 
it. 



8 



M. BENEDIKT, R. LENHARDT, AND J. WORRELL 



Proposition 3.8. Fix /c € N. Given u = uqUi . . . G T,^ , there exists G N such that for all 
n > N and any word w € T,^ with mi{w) = ran{w) = inf (u) it holds that u uqUi . . . UnW. 

Proof. Define a strictly increasing sequence of integers no < ni < . . . < inductively as 
follows. 

Let no be such that for all i > uq letter n, occurs infinitely often in u. For < s < A; 
let rig be such that ran(ti„^_j . . . = inf(u). Now define N := n^. 

Let n > N and let v := uqUi . . . UnW for some w such that inf(u;) = ran{w) = inf(n). 
We claim that for all < s < /c: 

(1) if z < n^ then Ts(u, i) = Ts{v, i); 

(2) if i,j > Us then Ts{u,i) = Ts{v,j) if Uj = vj. 

This claim entails the proposition. We prove the claim by induction on s. The base case 
s = is obvious. 

The induction step for Clause 1 is as follows. Suppose that i < Ug; we must show that 
Ts{u,i) = Ts{v,i). Certainly m = Vi since u and v agree in the first N letters. Similarly for 
all j < i we have Ts-i{u,j) = Ts-i{u,j) by Parts 1 and 2 of the induction hypothesis. Now 
for all j > i there exists j' > i such that uj = Vji and hence by Part 2 of the induction 
hypothesis rs_i(u, j) = Ts-i{v,j'). We conclude that Ts{u,i) = Ts{v,i) by Proposition 



3.2 



The induction step for Clause 2 is as follows. Suppose that i,j > Ug and m = vj; we 
must show that Ts{u,i) = Ts{v,j). We will again use Proposition |3.2[ Certainly for all 
i' > i there exists j' > j such that n^/ = Vj' and hence rs_i(n, i') = Ts-i{v,j'). Now let 
i' < i. If i' < Us then i' < j, Ui/ = Uj/ and hence Ts-i{u, i') = Ts-i{v, i'). Otherwise suppose 
Ug < i' < i. By definition of Ug there exists j', Ug-i < j' < n^ such that Uj/ = Vji. Then 
Ts-i{u,i') = Ts-i{u,j') by Clause 2 of the induction hypothesis. □ 



Combining Proposition 3.7 and Proposition |3.8[ we complete the proof of Proposition 



3.1, giving a slight strengthening of the conclusion for infinite words. 



Lemma 3.9. For any string u £ Ti^ and positive integer k there exists v G T,* with \v\ G 
20{\v\k) gy^^fi fiig^f y ^' jg^ infinitely many prefixes u' of u, and u ~fc vw^ , where w is a 
list of the letters occurring infinitely often in u. 

3.1. FO^ and temporal logic. We now examine the relationship between FO^ and UTL. 
Again we will be summarizing previous results while adding some new ones about the 
complexity of translation. 

As mentioned previously, Etessami, Vardi and Wilke |EVW02] have studied the expres- 
siveness and complexity of FO'^ on words. They show that FO^ has the same expressiveness 
as unary temporal logic UTL, giving a linear translation of UTL into FO^, and an expo- 
nential translation in the reverse direction. 

Lemma 3.10 ( |E VW02] ) . Every FO^ formula ip{x) can he converted to an equivalent UTL 
formula ip' with \ip'\ € 2'^(l'^l(''''^('^)"'"^)) and odp{(p') < 2qdp{(p). The translation runs in 
time polynomial in the size of the output. 

With regard to complexity, |EVW02] shows that satisfiability for FO^ over finite words 
or w-words is NEXP-complete. The NEXP upper bound follows immediately from their 



"small model" theorem (see Proposition 3.1 stated earlier). NEXP-hardness is by reduction 
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from a tiling problem. This reduction requires either the use of the successor predicate, 
or consideration of models where an arbitrary Boolean combination of predicates can hold, 
that is, they consider words over an alphabet of the form S = 2^^i'-^2, --.^'n}_ 

The NEXP-hardness result for FO^ [<] does not carry over from satisfiability to model 
checking since the collection of alphabet symbols that can appear in a word generated by 
the system being checked is bounded by the size of the system. However the complexity of 
model checking is polynomially related to the complexity of satisfiability when the latter 
is measured as a function of both formula size and alphabet size. Hence in the rest of the 
section we will deal with words over alphabet T, = {Pq, Pi, . . . , Pn}, i-e., in which a unique 
proposition holds in each position. We call this the unary alphabet restriction. 

One obvious approach to obtaining upper bounds for model checking F0^[<] would be 
to give a polynomial translation to TL[^,<$>], and use logic-to-automata translation for 
TL[0,^]- Without the unary alphabet restriction an exponential blow-up in translating 
from F0^[<] to TL[0,<$>] was shown necessary by Etessami, Vardi, and Wilke: 

Proposition 3.11 ( |EVW02] ) . There is a sequence {i^n)n>i of F0^[<] sentences over 
{Pi,P2, . . . ,Pn} of size O(n^) such that the shortest temporal logic formula equivalent to 
t/jn has size 

The sequence given in |EVW02] to prove the above theorem is 

= Vx Vy {/\{P{x) ^ Pi{y)) ^ (P„(x) o P„(y))). 

In particular, their proof does not apply under the unary alphabet restriction. However 
below we show that the exponential blow-up is necessary even in this restricted setting. Our 
proof is indirect; it uses the following result about extensions of FO^ with let definitions: 

Lemma 3.12. There is a sequence {^Pn)n>i o/FO^[<]Let sentences mentioning predicates 
{Pi, P2, ■ ■ . , Pn} such that the shortest model of under the unary alphabet restriction has 
size 2^(1'^"!). 

Proof. We define ifn as follows. 

ifn = Let Ri{x) be 3y (y < X A Pi{y)) in 

Let i?2(x) be3y{y<xA P2{y) A {Ri{x) o Ri{y))) in 

Let Rn{x) be 3y < X A P„(y) A /\ {Rk{x) o i?fc(y)) j 

in Vx /\ 3y I (-(Pi(x) ^ R,{y)) A l\{Rj{x) ^ Rj{y)) 
i=l \ j^i 

The body of the nested sequence of let definitions states that for all x and for all 1 < i < n 
there exists y such that the vector of formulas {Ri{x), R2{x), . . . , Rn{x)) has the same 
truth value as the vector (i?i(y), R2{y), . . . , Rn{y)) in all but position i. Hence the vector 
(iii(x), i?2(x), . . . , Rn{x)) must take all 2" possible truth values as x ranges over all positions 
in the word, i.e., any model of ipn must have length at least 2". 

We now claim that (fn is satisfiable. To show this, recursively define a sequence of words 
w^''^ over alphabet E = {Po, Pi, ... , Pn} by w^'^^ = e and = w'^^^ Pn-kw'^^\ where < 



10 



M. BENEDIKT, R. LENHARDT, AND J. WORRELL 



k < n. Finally write w = WuPq- Then the vector of truth values (iii(x), R2{x), ■ ■ ■ , Rn{x)) 
counts down from 2" — 1 to in binary as one moves along w. D 

In contrast, we show that basic temporal logic enhanced with let definitions has the 
small model property: 

Lemma 3.13. There is a polynomial poly such that every satisfiable TL[0,^]Lct formula 
(f has a model of size poly{\ip\). 

Proof. In |EVW02] Section 5], Etessami, Vardi, and Wilke prove a small model property 
for TL[0,'$>], which follows the same lines as the one given for FO^, but with polynomial 
rather than exponential bounds on sizes. Instead of using types based on quantifier-rank, 
the notion of type is based on the nesting of modalities; they thus look at modal fc-type, 
where k is the nesting of modalities in ip. It was shown how to collapse infinite w-words in 
order to get "smaller" cj-words with essentially the same type structure. Then in Lemma 
4 of |EVW02] it is shown that for each u G T,^ there are strings v, w such that the type 
of u at position is equal to the type of vw'^ at position and the length of both v and 
w is less than (t + 1)^, where t is number of types occurring along u (that is, a polynomial 
version to Proposition |3.1[ ). 

The type is determined by the predicate and the combination of temporal subformulas 
of ip holding at the given position. Each temporal subformula, i.e. subformula which starts 
with O or 0, can change its truth value at most once along the infinite word. Therefore 
there are at most polynomially many (in | S | and in number of temporal subformulas of (p) 
diff^erent combinations and so also types along u. 



Lemma 2.1 tells us that number of temporal subformulas of ip is linear in \ip\, and 
therefore the number of types t occurring along any word is polynomial in \ip\. Thus 
applying the above-mentioned type-collapsing argument of [EVW02] we conclude that there 
is a polynomial size model of D 

The small model property for TL[0, ^]Lct will allow the lifting of NP model-checking 
results to this language. Most relevant to our discussion of succinctness, it can be combined 
with the previous result to show that F0^[<] is succinct with respect to TL[0><$>]: 

Proposition 3.14. Even assuming the unary alphabet restriction, there is no polynomial 
translation from F0"[<] formulas to equivalent TL[<!y , <^]-formulas. 

Proof. Proof by contradiction. Assuming there were such a polynomial translation, we 
could apply it locally to the body of every let definition in an F02[<]Let formula. This 
would allow us to translate an FO^[<]Let form ula to a TL[05<$>]Let formula of polynomial 



size. Therefore it would follow from Lemma 3.13 that every FO [<]Let formula that is 



satisfiable has a polynomial sized model, which is a contradiction of Lemma 3.12, □ 



Proposition 3.14 shows that we cannot obtain better bounds for F0^[<] merely by 
translation to TL[0)^]- Weis jWeillj showed an NP-bound on satisfiability of F0^[<] 
under the unary alphabet restriction (compared to NEXP-completeness of satisfiability in 
the general case). His approach is to show that models realise only polynomially many 
types. We will later show that the approach of Weis can be extended to obtain complexity 
bounds for model checking F0^[<] that are as low as one could hope, i.e., that match 
the complexity bounds for the simplest temporal logic, TL[^,<$>]. We do so by building 
sufficiently small unambiguous Biichi automata for F0^[<] formulas. 
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4. Translations 

This section contains a key contribution of this paper — three logic-to-automata translations 
for UTL, FO^, and F02[<]. We will later use these translations to obtain upper complexity 
bounds for model checking both non-deterministic and probabilistic systems. As we will 
show, for most of the problems it is sufficient to translate a given formula to an unambiguous 
Biichi automaton. Our first translation produces such an automaton from a given UTL 
formula. This is then lifted to full FO^ via a standard syntactic transformation from FO^ 
to UTL. Our second translation goes directly from the stuffer-free fragment F0^[<] to 
unambiguous Biichi automata, and is used to obtain optimal bounds for this fragment. Our 
third translation constructs a deterministic parity automaton from an FO'^ formula. Having 
a deterministic automaton is necessary for solving two-player games and quantitative model 
checking of Markov decision processes. 

4.1. Translation I: Prom UTL to unambiguous Biichi automata. We begin with a 
translation that takes UTL formulas to Biichi automata. Combining this with the stan- 
dard syntactic transformation of FO^ to UTL, we obtain a translation from FO^ to Biichi 
automata. 

Recall from the preliminaries section that a Biichi automaton A is said to be determin- 
istic in the limit if all accepting states and their descendants are deterministic, and that A 
is unambiguous if each word has at most one accepting run. 

We will aim at the following result: 

Theorem 4.1. Let (p he a UTL formula over set of propositions V with operator depth n 
with respect to O and Q . Given an alphabet S C 2^, there is a family of at most 21*^1 Biichi 
automata {Ai}iQj such that (i) {w G T,''^ : w \= if} is the disjoint union of the languages 
L{Ai); (a) Ai has at most 0{\ip\\Tj\'^^^) states; (Hi) Ai is unambiguous and deterministic 
in the limit; (iv) there is a polynomial-time procedure that outputs A^ given input ip and 
index i £ L. 

We first outline the construction of the family {Ai}. Let ip he a formula of TL[0,^] 
over set of atomic propositions V. Following Wolper's construction |Wol01| . define cl{ip), 
the closure of p, to consist of all subformulas of (including ip) and their negations, where 
we identify -i-i?/; with ip. Furthermore, say that s C cl{(p) is a subformula type if (i) for 
each formula ip £ cl{p) precisely one of ij) and is a member of s\ (ii) ip £ s implies 
<C}ilj,0ip S s; (iii) ipi A Tp2 £ s iS ipi £ s and ■02 G s. Given subformula types s and t, write 
s ~ t if s and t agree on all formulas whose outermost connective is a temporal operator, 
i.e., for all formulas ^p we have OV' £ s iff O0 E t, and <$>'0 G s iff <$>'0 G t. Note that 
these types are different from the types based on modal depth considered before. 

Fix an alphabet S C 2^ and write tp^ for the set of subformula types s C cl{p) with 
s n P € S. In subsequent applications T, will arise as the set of propositional labels in a 
structure to be model checked. Following |Wol01| we define a generalised Biichi automaton 
A^ = {T,,S,So,A,X,J^) such that L{A^) = {w £ T."^ : {w,0) |= p}. The set of states is 
S = tp^, with the set Sq of initial states comprising those s £ tp^ such that (i) p £ s and 
(ii) <^'ip £ s ii and only if ^p £ s for any formula ip. The state labelling function A : S — )• S 
is defined by A(s) = sOP. The transition relation A consists of those pairs (s, t) such that 

(i) <$>V' G t iff either ip £ t or <^ip £ s; 

(ii) OV' ^ ^ ^iid ^ s implies OV' ^ *j 
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(iii) £ ^ implies -^C'Tp G t. 

The collection of accepting sets is = {F<^^ : <0">p ^ cl{ip)}, where F<^^ = {s : ip G 
s or OV' 

A run of on a word u £ T,^ yields a function / : N — t- 2'^^^'^\ Moreover it can be shown 
that if the run is accepting then for all formulas ^ £ cl{ip), V ^ /(^) =^ |= tp |Wol01t 

Lemma 2]. But since f{i) contains each subformula or its negation, we have ■0 ^ /(O 
and only if {u,i) \= ip for all ijj G cl{ip). We conclude that is unambiguous and accepts 
the language L{(p). The following lemma summarises some structural properties of the 
automaton A^. 

Lemma 4.2. Consider the automaton A^ as a directed graph with set of vertices S and 
set of edges A. Then (i) states s and t are in the same strongly connected component 
iff s t; (a) each strongly connected component has size at most \T,\; (iii) the dag of 
strongly connected components has depth at most \ip\ and outdegree at most 2^'^^; (iv) A^ 
is deterministic within each strongly connected component, i.e., given transitions {s,t) and 
(s, u) with s, t and u in the same strongly connected component, we have t = u if and only 
if\{t) = \{u). 

Proof, (i) If s ~ t then by definition of the transition relation A we have that (s,t) G A. 
Thus s and t are in the same connected component. Conversely, suppose that s and t are 
in the same connected component. By clauses (i) and (iii) in the definition of the transition 
relation A we have that <^?/'GsiffO'i/'^* ^md likewise -■OV' G s iff ^<0>ip G t. But for 
each formula G cl{(p) either s contains ip or its negation, and similarly for t; it follows 
that s ~ t. 

(ii) If s ~ t, then s = t if and only if A(s) = A(t). Thus the number of states in an 
sec is at most the number |S| of labels. 

(iii) Suppose that (s,t) G A is an edge connecting two distinct SCC's, i.e., s ^ t. 
Then there is a subformula OV' £ ^ such that -■OV' ^ Note that ^<0>ip lies in all states 
reachable from t under A. Since there at most \(p\ such subformulas, we conclude that the 
depth of the DAG of SCC's is at most \ip\. 

(iv) This follows immediately from (i). O 



We proceed to the proof of Theorem |4.1[ 

Proof. We first treat the case n = 0, i.e., (p does not mention O or 0. 

Let A^ = (S, 5, 5o, A, A, J^) be the automaton corresponding to (p, as defined above. 
For each path vr = Co, Ci, . . . , Cjt of SCC's in the SCC dag of A^ we define a sub-automaton 
At^ as follows. At^ has set of states St^ = Cq U Ci U • • • U C^; its set of initial states is SoCiStt] 
its transition relation is A^ = A n (5',^ x S'tt), i.e., the transition relation of A^ restricted 
to Stt] its collection of accepting states is -Fjr = {F CiCk ■ F £ F}. 



It follows from observations (ii) and (iii) in Lemma 4.2 that A,^ has at most Iv^HSl 
states, and from observation (iii) that there are at most such automata. Since A^ is 
unambiguous, each accepting run of A^ yields an accepting run of At^ for a unique path vr, 
and so the -L(^^) partition L{A^). 

Finally, is deterministic in the limit since all accepting states lie in a bottom 
strongly connected component, and all states in such a component are deterministic by 
Lemma |4.2riv). If we convert A^^ from a generalised Biichi automaton to an equivalent 
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Biichi automaton (using the construction from |Wol01j ). then the resulting automaton re- 
mains unambiguous and deterministic in the hmit. This transformation touches only the 
bottom strongly connected component of A.,^, whose size will become at most quadratic. 

This completes the proof in case n = 0. The general case can be handled by reduction to 
this case. A UTL formula if can be transformed to a normal form such that all next-time O 
and last-time operators are pushed inside the other Boolean and temporal operators. Now 
the formula can be regarded as a TL[0> ^] formula cp' over an extended set of propositions 
{O^P,Q^P : < i < n, P £ V}. Applying the case n = to we obtain a family of 
automata {^4'^} over alphabet S' = 2^ such that L{A^,) = IJ.L(A^), A'^ is unambiguous 
and deterministic in the limit, and A'- has at most 0(|(y9'||$]'|) = 0(|(/?||S|'^) states. 

Now we can construct a deterministic transducer T with states that transforms 
(in the natural way) an w-word over alphabet S into an w-word over alphabet S'. Such 
a machine can be made deterministic by having T produce its output n positions behind 
the input. To do this we maintain an n-place buffer in the states of T, which requires ISI*^ 
states. 

We construct automaton Ai over alphabet S by composing A[ with T, i.e., by synchro- 
nising the output of T with the input of A'^. The number of states of the composition is 
the product of the number of states of A'- and T which are consistent with respect to their 
label in S'. Thus the product has at most Odt^llSl""^^) states. 

This completes the proof of Theorem |4.1[ □ 



From Theorem 4.1 we can get a translation of FO^ to automata with bounds as stated 
below: 

Theorem 4.3. Given an FO^ formula ip, there is a collection 0^22''°'''^''^" generalised Biichi 
automata Ai, each of size at most 2*"''^(l'^l) such that the languages they accept partition the 
language {w £ S"^ : w \= (p}. Moreover, each automaton Ai is unambiguous and can be 
constructed by a non-deterministic Turing machine in polynomial time in its size. 



Proof. First we apply Lemma 3.10 to translate the FO formula ip to an equivalent UTL 



formula ip' . We then apply Theorem 4.1 to ip' , noting that the size of (p' is exponential in 



the size of ip, while the operator depth of (p' is polynomial in the quantifier depth of (p. 



Finally, we apply Theorem 4.1 to ip'. □ 



4.2. Translation II: Prom F0^[<] to unambiguous Biichi automata. The previous 
translation via UTL will be useful for giving bounds on verifying both UTL and FO^. 
However it does not give insight into the sublanguage F0^[<]. We will thus give another 
translation specific to this fragment. The main idea for getting upper bounds on verification 
problems for F0^[<] will be to show that for any F0^[<] formula if, the number of one- 
variable subformula types realised along a finite or infinite word is polynomial in the size of 
ip. Informally these subformula types are the collections of one-variable subformulas of ip 
that might hold at a given position. Note that the types we consider here are collections of 
F0^[<] formulas, not temporal logic formulas as in the last section. Also note the contrast 
with the /c- types of Proposition |3.1[ which consider all formulas of a given quantifier rank. 

Recall that the domain of a word u G S* U S"^ is the set dom(tt) = {i€N:0<z< \u\} 
of positions in u. Given an FO^ [<]-formula ip, let cl{ip) denote the set of all subformulas 
of ip with at most one free variable (including atomic predicates). Given a finite or infinite 
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word u £ T,* UT,^, a position i G dom(ti), we define the subformula type of u at position i 
to be the set of F0^[<] formulas 

r(n, i) = {tp : ip £ cl{(f) and (n, i) |= ^} . 

We have omitted f in this notation since it wiU be fixed for the remainder of the proof. 

Few Types Property for F0^[<]. We wih base our result on the following theorem 
of Weis [Weillj . showing that F0^[<] formulas divide a finite word into a small number of 
segments based on subformula type: 

Proposition 4.4 ( |Weillj ). Let be an F0'^[<] -formula. A string n G S* can be written 
u = vi . . . Vn, where Vi G T,* , n is polynomial in \{p\ and and for any two positions i,j 
lying in the same factor Vk having the same symbol, T{u,i) = T{u,j). 

We will need an extension of this result to infinite words: 

Proposition 4.5. Let ip be an F0'^[<] -formula. A string u £ can be written u = 
vi . . .Vn, where Vk € S* for k < n and Vn € S'^, n is polynomial in \ip\ and \T,\, and for 
any two positions i,j lying within the same factor and having the same symbol we have 
T{u,i) = T{u,j). 

Proof. We note that for any u G T,^, from some position onwards, the subformula type is 



determined only by the current symbol. In fact, the proof of Proposition 3.8 shows that we 
have u = vw for some prefix v € T,* of u and w £ Ti^ such that for any two positions i,j of 
vw such that i,j > \v\ having the same symbol T{vw,i) = T{vw,j). 



Given an infinite u, we can take a finite prefix v as above and apply Proposition 4.4 to 
it, adding on the infinite interval w as one additional member of the partition. Now if i and 
j are in the final partition, then agreement on the same symbol determines the entire set of 
formulas, and hence we are done. Otherwise, fix any two positions i,j < \v\ in u with the 
symbol a G E holding at both i and j, and lying in the same partition within v. We claim 
that the subformula types T{u,i) and T{u,j) contain the same set of formulas. An atomic 
predicate ip £ cl{ip) holds at position i iff it holds at j by assumption, since there is only 
one symbol true at each position. Positions i and j then by assumption satisfy the same 
subformula type within v. But using the hypothesis on v we can easily see inductively that 
a subformula holds on a position within v iff it holds at that position within vw. D 

We now present a result showing that the few subformula types property can be used 
to get a better translation to automata: 

Theorem 4.6. Assume the unary alphabet restriction. Then given an F0^[<] formula 
ip, there is a collection o/ 2^'''^(l'^l'l^l^ generalised Biichi automata Ai (each of polynomial 
size in \ip\ and \T,\) such that the languages they accept are disjoint and the union of these 
languages is exactly {w £ : w \= ip} . Moreover, each automaton Ai is unambiguous and 
deterministic in the limit and can be constructed by a non- deterministic Turing machine in 
polynomial-time. 

Proof. We say that r C cl{ip) is a subformula pre-type if: (i) if ^pi A ip2 £ cl{ip), then 
ipi A ip2 £ T iS (pi £ T and (^2 £ t', (ii) if Vi V 9^2 £ cl{p>), then ipi y ip2 £ t iE pi £ t 01 
f2 £ T] (iii) if 6 cl{ip), then S r iff -0 r. 

This notion is similar to the notion of "subformula type of a node" used in the prior 
results, except that a collection of formulas satisfying the above property may not be con- 
sistent, since the semantics of existential quantifiers is not taken into account. 
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In general the formulas in a (subformula) pre-type r can have either x or y as free 
variables. We write t{x) for the subformula pre-type obtained by interchanging x and y in 
all formulas in r with y as free variable. Thus all formulas in t{x) have free variable x. We 
similarly define T{y). 

An order formula is an atomic formula 

a ::= x < y \ y < x \ x = y . 

Given m, n G N let am,n denote the unique order formula satisfied by the valuation x, y i— )• 
m, n. 

Given a pair of pre- types ti,T2, an order formula a, and a subformula of 99, we 
write ri(j;), r2(y), a |= 6{x,y) to denote that when 6 is transformed by replacing top-level 
subformulas by their truth values as specified by ti{x), T2{y), or a, then the resulting 
Boolean combination evaluates to true. Note that this implies that if word w and positions 
i,j satisfy ti(x) U T2{y) U {a}, then they also satisfy 6. 

A closure labelling is a function / : N ^ 2'=^('^) such that 

(1) /(n) is a pre-type for each n G N and 

(2) for each n E N, if 3y0 € cl{ip) then 3y6 € f{n) iff there exists m G N such that 
f{n){x),f{m){y),an,m |= 6*- 

It is easy to see that an cj-word if : N — )■ S has a unique extension to a closure labelling 
/ : N ^ 2'='('^). Namely, / is defined by /(n) = {V' G d{ip) :w,n^ i/j}. 
We now define a generalised Biichi automaton corresponding to (p. 

Definition 4.7. The alphabet of A^p is S, and the other components of A^p are as follows: 
States. The states of A^ are tuples (s,r, t), where r C 0/(93) is a pre-type and 
s,t Q 2'^'^'^) are sets of pre-types of size at most where p is the polynomial 



from Proposition 4.5 such that the following consistency condition holds: for each formula 
3y9 G T we have that either r(a;),r(y),x = y \= 6, t{x),t' (y), x < y \= 9 for some t' £ t, 
or t' {y),T{x),y < x \= 9 for some t' G s. (This condition corresponds to the second clause 
in the definition of closure labelling.) Informally, a state consists of an assertion about the 
subformula pre-types seen in the past, the current subformula pre-type, and the subformula 
pre-types to be seen in the future. 

Initial State. A state (s,r, t) is initial if s = and ip G t. 

Accepting States. There is a set of accepting states Fr for each pre-type r. We have 
(s, t' , t) G Ft- if and only if r = r' or r t. 

Transitions. For each a G S there is an a-labelled transition from (s,r, t) to (s',T',t') 
iff (i) for the unique proposition Pi{x) in r. Pi = a; (ii) s' = sL) {r}; (iii) r' G t; (iv) either 
t' =tor t' = t \ {r'}. 



The following proposition, whose proof follows straightforwardly from Proposition 4.5 
shows that the automaton captures the formula: 

Proposition 4.8. // (sq, tq, to)) (^ii ''"ij *i)) (^2, '^2, ^2)) • • • is an accepting run of A^, then 
the function / : N — )■ 2'^'^'^^ defined by f{n) = Tn is a closure labelling. Moreover every 
closure labelling f such that (p G /(O) arises from a run of A^ in this manner. 

We now analyze the automaton A^. Because of the polynomial restriction on the num- 
ber of pre-types, the automaton has at most exponentially many states. But by Proposi- 



tion 4.5 , any accepting run goes through only polynomially many states. For every path tt in 
the DAG of strongly-connected components, we take the subautomaton A-,^ of A^p obtained 
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by restricting to the components in this path. We claim that this is the required decom- 
position of Aip. Note that an NP machine can construct these restrictions by iteratively 
making choices of successor components that are strictly lower in the DAG. Clearly the 
automata corresponding to distinct paths accept disjoint languages, since they correspond 
to different collections of pre-types holding in the word. One can show that for any word 
satisfying the formula, the unique accepting run is the one in which the state at a position 
corresponds to the pre-types seen before the positions, the pre-type seen at the position, 
and the pre-types seen after the position. In particular, this shows that each automaton 
is unambiguous. Finally, because the only nondeterministic choice is whether to leave an 
sec or not, upon reaching the bottom SCC the automaton is deterministic — hence each 



automaton is deterministic in the limit. Thus this decomposition witnesses Theorem 4.6 D 



The above translation of F0^[<] formulas to unambiguous Biichi automata can be ex- 
tended to handle formulas with successor, i.e., the full logic FO^, at the same time removing 
the unary alphabet restriction. Given an FO^ formula ip over set of predicates V, we can 
consider an "equivalent" F0^[<] formula (f' over a set of new predicates 21*^1 I^L Intuitively 
each predicate in V' specifies the truth values of all predicates in 7^ in a neighbourhood of 



radius \ip\ around the current position. Applying Theorem 4.6 to we obtain a collection 



of double-exponentially many automata Ai, each of size exponential size in ip and S. Thus, 



we get a weaker version of Theorem 4.3 of the previous subsection, in which the size bound 
on the component automata has an exponential dependence on the alphabet as well as the 
formula size. 

4.3. Translation III: Prom FO^ to deterministic parity automata. While the previ- 
ous translations are useful for relating FO^ to unambiguous automata, for some problems 
it is useful to have deterministic automata. We now give a translation of FO^ formulas 
to "small" deterministic parity automata. We give the translation first for the fragment 
F02[<] without successor and show later how to handle the full logic. Specifically, we will 
show: 

Theorem 4.9. Given an F0^[<] formula ip over set of predicates V with quantifier depth k, 
there exists a deterministic parity automaton A^p accepting the language L({p) such that Ay, 
has 22°"''''' states, 20^) 

priorities, and can be computed from ip in time \(p\'^^^^ -2^ ^ . 



The definition of the automaton A^, in Theorem 4.9 relies on the small-model property. 



as stated in Proposition 3.1 By Lemma 3.9, to know whether u G S'^ satisfies an F02[< 



formula of quantifier depth k it suffices to know some A:-type such that infinitely many 
prefixes of u have that type, as well as which letters occur infinitely often in u. We will 
translate cp to a deterministic parity automaton A^ that detects this information. As Aip 
reads an input stri ng u it stores a representative of the /c-type of the prefix read so far. 



By Proposition 3.1 i) the number of such representatives is bounded by 2^'^''''''°' . Applying 



Lemma [3. 9[ we use a parity acceptance condition to determine whether u satisfies ip, based 
on which representatives and input letters occur infinitely often. 

We are now ready to formally define A,p. To this end, define the last appearance record 
of a finite string u = mq • • • ""n G S* to be the substring LAR(tt) := Ui^Ui^ ■ ■ ■ Ui^ such that 
for all k £ dom(n) there exists a unique ij > k such that = Uk- Thus we obtain LAR(n) 
from u by keeping only the last occurrence of each symbol from u. Write LAR(S) for the 
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set {LAR(m) : n E S*} of all possible last appearance records. Recall also the set of strings 



Rep;j,(S) from Corollary 3.6 that represent the different A;-types of strings in S* 



Definition 4.10. Let ip{x) be an FO^ [<]-formula of quantifier depth k. We define a deter- 
ministic parity automaton A^p as follows. 

• has set of states Repfc(i;) x LAR(S) x {0, 1, . . . , 

• The initial state is (e,e, 0). 

• The transition function maps a state (s, I, i), where i = ii . . 



to the unique state {t 
in i and otherwise ij' = a. 
The set of priorities is 0, 1, 
The priority of state {s,£,i 

pr{s, £, i) = 



such that sa t, i' = LAR(^a), j 



and input letter a £ T, 
= if a does not occur 



..,2|S| + 1. 

where £ = ii£2 



J is given by 

if {sii^...ijr,0) ^ip 



+ 1 otherwise. 



It follows from Proposition 3.7 that in a run of Aip on a finite word u = uqUi . . .Un G S* 
the last state {s, i, i) is such that s has the same /c-type as u. Also we note that i is the 
LAR of u and i is the position in the previous LAR of Un- 
The following two results prove Theorem |4.9[ 

Proposition 4.11. L{A^) = {u G S'^ : (u, 0) |= ip}. 



Proof. Let u G S'^ and let be as in Proposition |3.8[ Suppose that the highest infinitely 
often occurring priority in a run of A^p on u is even. Then there exists n > N such that A^ 
is in state (s, i, i) after reading uqUi . . .Un, where £ = £i£2 ■ ■ ■£j, {ii, . . . , ij} = mf{u) and 
{si£i...£jr,0)^^. Now 

u ~fc uqui . . . Un{ii ■ ■ ■ £j)^ by Proposition 3^ 

~fc s{£i . . . £j)^ by Proposition 3/7 . 

We conclude that (u, 0) |= ip. 

Similarly we can show that if the highest infinitely often occurring priority in a run of 



Aip on u is odd then (n, 0) ^ (p. 



□ 



Proposition 4.12. If p over set of monadic predicates V has quantifier depth k, then A^ 
has number of states at most 22°*''''''' and can be computed from p in time \p\'~^^^^ ■ 22°*'''''''\ 



Proof. The set of states Rep^ (S) has size at most 2 



20(\V{k) 



at most 2^°*''''''' by Corollary 



3.6 



and can be constructed in time 
We can establish the existence of a transition between 

Finally we can compute 



3.5 



any pair of states of A^ in time at most 2'^(l^l'^) by Proposition 
the priority of a state {s,£,i) by model checking p on a lasso of length at most 2*^(1^1'^^, 
which can be done in time I 2^(1^1*=). □ 

Extension to FO^ with su cces sor. We no w ex tend to successor using the same ap- 
proach as in the proof of Theorem 4.1 By Lemma 3.10 , given an FO^ formula p> of quantifier 



By Lemma 3 

depth k there is an equivalent UTL formula p' of at most exponential size and operator depth 
at most 2k. Moreover, p' can be transformed to a normal form such that all next-time O and 
last-time operators are pushed inside the other operators. Again, we consider p' also as 
a TL[0> <$>] -formula over an extended set of predicates V = {O^Pj,Q^Pj \ Pj £ P,i < k}. 
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By a straightforward transformation we get an equivalent F0^[<] formula ^p" over P' . Over- 
all, this transformation creates exponentially larger formulas, but the quantifier depth is 
only doubled and the set of predicates is quadratic. Applying Theorem 4.9 for Lp" over set 
of predicates V' gives: 

Theorem 4.13. Given an FO^ formula ip with quantifier depth k, there is a deterministic 

parity automaton having 2^*^** states and 2^^^^'^^^ priorities that accepts the language 
L{p). 



5. Models considered 

Next we collect together definitions of the various different types of state machine that we 
consider in this paper. For non-deterministic machines we will be interested in the existence 
of an accepting path through the machine that satisfies a formula, while for probabilistic 
models we want to know the probability of such paths. 

Kripke Structures, Hierarchical and Recursive State Machines. Our most 
basic model of non-deterministic computation is a Kripke structure, which is just a graph 
with an additional set of nodes (the initial states), and a labelling of nodes with a subset 
of a collection of propositions. The behavior represented by such a structure is the set of 
paths through the graph, where paths can be seen as w-words. 

We will look also at more expressive and succinct structures for representing behaviours. 
A recursive state machine (RSM) M over a set of propositions V is given by a tuple 
(Mi,...,Mfc) where each component state machine Mj = {Nj U Bi,Yi, Xi, Eui, Exi, 6i) 
contains 

• a set Ni of nodes and a disjoint set Bi of boxes; 

• an indexing function Yi : Bi {1, . . . ,k} that assigns to every box an index of one of the 
component machines. Mi, . . . , M^; 

• a labelling function Xi : Ni 2^; 

• a set of entry nodes Eui C Ni and a set of exit nodes Exi C Ni] 

• a transition relation 6i, where transitions are of the form {u,v) where the source u is 
either a node of A'^j, or a pair (6, x), where 6 is a box in Bi and x is an exit node in Exj 
for j = Yi{b). We require that the destination v be either a node in Ni or a pair (6, e), 
where 6 is a box in Bi and e is an entry node in Enj for j = 1^(6). 

Informally, an RSM represents behaviors that can transition through a box into the entry 
node of the machine called by the box, and can transition via an exit node back to the calling 
box, as with function calls. The semantics can be found in ABE^OS]. A hierarchical state 



machine (HSM) is an RSM in which the dependency relation between boxes is acyclic. 
HSMs have the same expressiveness as fiat state machines, but can be exponentially more 
succinct. 

Markov Chains. The basic probabilistic model corresponding to a Kripke structure 
is a (labelled) Markov chain, specified as A4 = (S, X, V, E, M, p), consisting of an alphabet 
S, a set X of states; a valuation y : X — )■ S; a set E <^ X x X edges; a transition 
probability M^y for each pair of states {x,y) £ E such that for each state x, Yly = 1; 
an initial probability distribution p on the set of states X. 

A Markov chain defines a probability distribution on trajectories — paths through the 
chain. Given a language L C S"^, we denote by Pm{L) the probability of the set of 



TWO VARIABLE VS. LINEAR TEMPORAL LOGIC IN MODEL CHECKING AND GAMES 



19 



trajectories of A4 whose image under V lies in L. We consider the complexity of the 
following model checking problem: Given a Markov chain Ai and an LTL- or FO^-formula 
(p, calculate Pm{L{^p)). There is a decision version of this problem that asks whether this 
probability exceeds a given rational threshold. 

Recursive Markov Chains. Recursive Markov chains (RMCs) are the analog of 
RSMs in the probabilistic context. They are defined as RSMs, except that the transition 
relation consists of triples {u,pu,v^v) where u and v are as with RSMs, and the pu,v are 
non-negative reals with Ti^pu^v = 1 or for every u. As with Markov chains, these define 
a probability distribution on trajectories, but now trajectories are paths which must obey 
the box-entry/box-exit discipline of an RSM. The semantics of an RMC can be found in 
|EY05j . A hierarchical Markov chain (HMC) is the probabilistic analog of an HSM, that is, 
an RMC in which the calling graph is acyclic. An HMC can be converted to an ordinary 
Markov chain via unfolding, possibly incurring an exponential blow-up. An example of an 
RMC is shown in Figure [2] 
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Figure 2: A sample Recursive Markov Chain 



Markov Decision Processes. We will also deal with verification problems related 
to control of a probabilistic process by a scheduler. A Markov decision process (MDP) 
Ai = (S, X, N, R, V, E, M, p) consists of an alphabet S, a set X of states, which is partitioned 
into a set N of non- deterministic states and a set R of randomising states; a valuation 
V : X ^ T,, a set E X x X of edges, a transition probability Mxy for each pair of states 
{x,y) £ E, X G R such that Yly-^^xy = 1; an initial probability distribution p. This model 
is considered in |CY95j under the name Concurrent Markov chain. 

We can view non-deterministic states as being controlled by the scheduler, which given 
a trajectory leading to a non-deterministic state s chooses a transition out of s. There are 
two basic qualitative model checking problems: the universal problem (V) asks that a given 
formula be satisfied with probability one for all schedulers; the existential problem (3) asks 
that the formula be satisfied with probability one for some scheduler. The latter corresponds 
to the problem of designing a system that behaves correctly in a probabilistic environment. 
In the quantitative model checking problem, we ask for the maximal probability for the 
formula to be satisfied on a given MDP when the scheduler chooses optimal moves in the 
non-deterministic states. 

Two-player Games. A two-player game G = (S, X, Xi, X2,V, E, xq) consists of an 
alphabet S; a set X of states, which is partitioned into a set Xi of states controlled by 
Player I and a set X2 controlled by Player U; a set of C X x X of transitions; a 
valuation 1/ : X — )• S; an initial state xq. 



20 



M. BENEDIKT, R. LENHARDT, AND J. WORRELL 



The game starts in the initial state and then the player who controls the current state, 
taking into account the whole history of the game, chooses one of the possible transitions. 
The verification problem of interest is whether Player I has a strategy such that for all 
infinite plays the induced infinite word u £ T,'^ satisfies a given formula ip. 

Stochastic Two-player Games. A Stochastic two-player game (2 ^-player game) 
G = {X, Xi, X2, R, V, E, M,pq) consists of a set X of states, which is partitioned into a set 
Xi of states controlled by the first player, a set X2 controlled by the second player and a 
set R of randomising states; a valuation V : X ^ T,; a set of E <^ X x X transitions, a 
transition probability M^y for each pair of states G E, x G R such that Y^yMxy = 1; 

an initial probability distribution p. See Figure [3] for an example. 

The universal (V) qualitative model checking problem asks if the first player can enforce 
that the infinite word u, induced by the path through the game, satisfies f with probability 
one. 




Figure 3: A sample Stochastic Two-player Game. Diamonds are states of the first player, 
squares are states of the second player and circles represent randomising states. 



6. Verifying non-deterministic systems 

Model checking for traditional Kripke Structures is fairly well-understood. All of our logics 
subsume prepositional logic, and the model checking problems we deal with generalise 
propositional satisfiability — hence they are all NP-hard. LTL and UTL are both PSPACE- 
complete [SC82j, while (TL[0,^]) is NP-complete. 

Translation I shows how to convert an FO^ formula to a union of exponential sized 
automata. A NEXPTIME algorithm can guess such an automaton, take its product with a 
given Kripke Structure, and then determine non-emptiness of the resulting product. Cou- 
pled with the hardness argument in |EVW02] , this gives an alternative proof of the result 
of Etessami, Vardi, and Wilke: 

Theorem 6.1. |EVW02j FO^ model-checking is complete for NEXPTIME. 

Below we extend these results to give a comparison of the complexity of model checking 
for recursive state machines and two-player games, applying all of the translations in the 
previous section. 
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6.1. Recursive State Machines. Using Translation II, we sliow tliat F0^[<] model check- 
ing can be done as efficiently as for TL[0, ^] on non-deterministic systems, and in partic- 
ular for RSMs. 

Proposition 6.2. Model checking F0^[<] properties on Kripke structures, hierarchical and 
recursive state machines is in NP. 

Proof. We give the upper bound for RSMs only, since the other classes are special cases. We 
describe an NP algorithm that checks satisfiability of an F0^[<] sentence f on the language 
of RSM Ai. Model checking the structure involves only combinations of propositions oc- 
curring in the structure, and hence by expanding out these combinations explicitly, we can 
assume that the unary alphabet restriction holds. Thus we can apply Translation II, from 



F0^[<] to Biichi Automata, Theorem 4.6, It suffices to check that one of the automata Ai 
produced by the translation accepts a word produced by We can thus guess such an Ai 
and can then check intersection of Ai with Ai in polynomial time, by forming the product 
and checking that we can reach an accepting bottom strongly connected component. This 
reachability analysis can be done efficiently using the "summary edge construction" — see, 
e.g., jABE+n5| . □ 

In the same way, we can obtain the result for model checking full FO^ on RSMs, but 



now using the FO^ to automata translation in Translation 1, Theorem 4.3 Again we guess 
an automata Ai, which is now of exponential size. Thus we have: 

Proposition 6.3. FO^ model checking of RSMs can be done in NEXPTIME. 

This result matches the known result for ordinary Kripke structures. 

6.2. Two-player games with FO^ winning condition. Two-player games are known to 
be in 2EXPTIME for LTL [PR89]. We now show that the same is true for FO'^, making use 
of Translation III in the previous section, which translates to deterministic parity automata. 
We also utilise the fact that a parity game with n vertices, m edges and d priorities can be 
solved in time 0{dmn'^) | JurOO| . 

From these two results we easily conclude the 2EXPTIME upper bound: 

Proposition 6.4. Two-player games with FO^ winning conditions are solvable in 2EXP- 
TIME. 



Proof. Using Theorem 4.13[ we construct in 2EXPTIME a deterministic parity automaton 



for the FO^ formula 99 with doubly exponentially many states and at most exponentially 
many priorities. By taking the product of this automaton with the graph of the game, 
we get a parity game with doubly exponentially many states but only exponentially many 
priorities. (In fact if we define the automaton over an alphabet S C 2^ containing only sets 
of propositions that occur as labels of states in the game, then polynomially many priorities 
suffice.) We can then determine the winner in double exponential time, again applying the 
0{dmn'^) bound for solving games of |JurOO| mentioned above. □ 
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Combining this with the result by Alur, La Torre, and Madhusudan, who showed that 
two-player games are 2EXPTIME-hard |ATM03j already for the simplest TL[0>, along 
with the fact that we can convert UTL formula to FO^ formula in polynomial time, we get 
2EXPTIME-completeness : 

Corollary 6.5. Deciding two-player games with FO^ winning conditions is complete for 
2EXPTIME. 

The table below summarises both the known results and the results from this paper (in 
bold) concerning non-deterministic systems. All bounds are tight. 





TL[0,^] 


UTL 


F0^[<] 


FO^ 


LTL 


Kripke Structure 


NP 


PSPACE 


NP 


NEXP 


PSPACE 


HSM 


NP 


PSPACE 


NP 


NEXP 


PSPACE 


RSM 


NP 


EXP 


NP 


NEXP 


EXP 


Two pi. games 


2EXP 


2EXP 


2EXP 


2EXP 


2EXP 



The PSPACE bound for model checking LTL on HSMs follows by expanding the HSMs 
to 'flat' Kripke structures and recalling that model checking LTL on Kripke structures 
can be done in space polynomial in the logarithm of the model size. Additionally, the 
complexity of model checking UTL and LTL on RSMs is EXPTIME-complete |BEM97j . 
and model checking TL[<>,^] on RSMs is NP-complete |LTP07j . 

7. Verifying probabilistic systems 

We now turn to probabilistic systems. Here we will make use of two key properties of 
the automata produced by the first two translations — unambiguity and determinism in the 
limit. We will need two lemmas, which show that the complexity bounds for model checking 
unambiguous Biichi automata on various probabilistic systems are the same as the bounds 
for deterministic Biichi automata on these systems. First, following [CSS03j . we note the 
following property of unambiguous automata: 

Lemma 7.1. Given a Markov chain A4 = (S, X,V, E, M, p) and a generalised Biichi au- 
tomaton A = (S, 5", 5*0, A, A, -F) that is unambiguous, Pm{L{A)) can be computed in time 
polynomial in Ai and A. 

Proof. We define a directed graph M(^A representing the synchronised product of and A. 
The vertices of M<^A are pairs {x,s) G X x S with matching propositional labels, i.e., such 
that V{x) = A(s); the set of directed edges is {{{x , s) , {y , t)) : {x,y) £ E and (s,t) G A}. 
We say that a strongly connected component (SCC) oi M. A \s accepting if (i) for each 
set of accepting states F £ F it contains a pair (x, s) with s £ F and (ii) for each pair (x, s) 
and each transition G E, there exists G A such that {y,t) is in the same SCC 

as (x, s). This guarantees that we can stay in the SCC and visit each of its states infinitely 
often. 

Let L{A, s) denote the set of words accepted by A starting in state s. For each vertex 
(x,s) of M. 1^ A we have a variable ^x,s representing the probability P_\4^xiL{A, s)) of all 
runs of M starting in state x that are in L(A,s). These probabilities can be computed as 
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the unique solution of the following linear system of equations: 
£^x,s = 1 {x, s) in an accepting SCC 
£^x,s = (x, s) in a non-accepting SCC 

(.x,s = ^ ^ ■ iy^t otherwise. 

(^,t)eA y:V(y)=\{t) 

The correctness of the third equation follows from the following calculation: 

Pm,,(L(As)) = PmA U \{s)-L{A,t)) 

{s,t)eA 

= P^^xi^{s) • L{A,t)) (since A is unambiguous) 

(s,t)eA 

= E E Mxy.PM,yiL{A,y)). □ 

(s,t)eA y:V{y)=\{t) 

For an RMC , we can compute reachability probabilities q(^u,ex) of exiting a component 
Mi starting at state u £ Vi going to exit ex E Exi. Etessami and Yannakakis |EY05] show 
that these probabilities are the unique solution of a system of non-linear equations which 
can be found in polynomial space using a decision procedure for the existential theory of 
the reals. Following [EY05j for every vertex u £ Vi we let ne{u) = 1 — XlexeExi Qiu,ex) 
be the probability that a trajectory beginning from node u never exits the component Mj 
of u. Etessami and Yannakakis fYE05] also show that one can check properties specified 
by deterministic Biichi automata in PSPACE, while for non-deterministic Biichi automata 
they give a bound of EXPSPACE. Thus the prior results would give a bound of EXPSPACE 
for UTL and 2EXPSPACE for FO^. We will improve upon both these bounds. We observe 
that the technique of |YE05j can be used to check properties specified by non-deterministic 
Biichi automata that are unambiguous in the same complexity as deterministic ones. This 
will then allow us to apply our logic-to-automata translations. 

Proposition 7.2. Given an unambiguous Biichi automaton A and a RMC M, we can 

compute the probability that A accepts a trajectory of M in PSPACE. 

Proof. Let A be an unambiguous Biichi automaton with set of states Q, transition function 
A and labelling function A. Let A4 be an RMC with valuation V. We define a product 
RMC Ai A with component and call structure coming from A4 whose states are pairs 
(x, s), with X a state of A4 and s a state of A such that V{x) = X{s) (i.e., x and s have the 
same label). Such a pair {x, s) is accepting if s is an accepting state of A. A run through 
the product chain is accepting if at least one of the accepting states is visited infinitely 
often. Note that a path through Ai may expand to several runs in M A since A is 
non-deterministic. 

For each i, for each vertex x £ Vi, exit ex G Exi and states s,t £ Q we define p(x, s — )■ 
ex, t) to be the probability that a trajectory in RMC M that begins from a configuration 
with state x and some non-empty context (i.e. not at top-level) expands to an accepting 
run in Ai (Si A from (x, s) to (ex, t). 

Just as in the case of deterministic automata, we can compute p{x, s — >• ex, t) as the 
solution of the following system of non-linear equations: 
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If X G is not entrance of the box we have: 

s — )• ex, t) = Mxx' p{x' , s' ^ ex,t) 

x':ix,M^^i,x')£Si s':is,s')eAAX(s')=Vix') 

If X G is entrance of the box b £ Bi then we include the equations: 

p{x,s^ex,t)= p{{b,en),s ^ {b,exj),s')p{{b,exj),s' ^ ex,t) 
j,s'eQ 

where p{{b, en), s — )• (6, exj), s') = p(eny.(fe), s — )• exj, s') and exj G Exy^ib)- 

The justification for these equations is as foUows. Since A is unambiguous, each tra- 
jectory of M expands to at most one accepting run oi M ® A. Thus in summing over 
automaton states s' in the two equations above we are summing probabihties over disjoint 
events which correctly gives us the probability of the union of these events. 

We now explain how these probabilities can be used to compute the probability of 
acceptance. We assume without loss of generality that the transition function of A is total. 

We construct a finite-state summary chain for the product M. ® A exactly as in the 
case of deterministic automata |YE05] . For each component Mi of vertex x of Mj, exit 
ex G Exi and for each pair of states s,t oi A the probability to transition from (x, s) to 
(ex, t) in the summary chain is calculated from p(x, s — )• ex, t) after adjusting for probability 
ne(x) that M never exits Mj starting at vertex x. Note that since automaton A is non- 
blocking, the probability of never exiting the current component oi A starting at (x, s) 
is the same as ne(x) (the probability of never exiting the current component from vertex x 
in the RMC M alone). 

To summarise, we first compute reachability probabilities q(^u,ex) ^^(^ probabilities ne{u) 
for the RMC Ai. Then we consider the product Ai A and solve a system of non-linear 
equations to compute the probabilities of summary transitions p{x,s — )■ ex,t). From these 
data we build the summary chain, identify accepting SCCs and compute the resulting 
probabilities in the same way as in |YEn5j . All these steps can be expressed clS Si formula 
and its truth value can be decided using existential theory of the reals in PSPACE. □ 



7.1. Markov chains. We are now ready to prove a new bound for the model checking prob- 
lem on our most basic probabilistic system, Markov chains. Courcoubetis and Yannakakis 
|CY95j showed that one can determine if an LTL formula holds with non-zero probability 
in a Markov chain in PSPACE. This gives a PSPACE upper bound for TL[^,<$>] and an 
EXPSPACE upper bound for FO^. We will show how to get better bounds, even in the 
quantitative case, using the logic-to-automata translations. 

Proposition 7.3. Model checking TL[^,<$>] or F0^[<] on Markov chains is in #P. 



4.1 



Proof. Let fhe a TL[0, <$>] or F0 ^[<] formula and Ai a Markov chain. Using Theorem 
in case of TL[0)<3>] ^'iid Theorem 4.6 in case of F0^[<], we have that for formula ip there 



is a family {Ai} comprising at most 

2Poiy{M,\T.\) unambi guous generalised Biichi automata, 
whose languages partition {w £ T,^ : w \= ip}. Moreover, each Ai has at most states 



and can be generated in polynomial time from ip and index i. By Lemma 7.1 we can further 
compute the probability pi of A4 satisfying Ai in polynomial time in the sizes of Ai and Ai. 
Since each pi is computable in polynomial time we can determine pi in □ 
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Proposition 7.4. The threshold problem for model checking FO^ on Markov chains is in 
PEXP. 



Proof. The result follows by the same argument as in Proposition 7.3, as we are essentially 



in the same situation, but now by Theorem 4.3 we have a collection of doubly-exponentially 
many automata, each of exponential size. D 



7.2. Hierarchical and Recursive Markov chains. Similarly, we get the following results 
for recursive Markov chains (and in particular for hierarchical Markov chains): 

Proposition 7.5. The probability of a TL[0, ^] or F0^[<] formula holding on a recursive 
Markov chain can be computed in PSPACE. 



Proof. By Theorem 4.1 in case of TL[Oi ^] and by Theorem 4.6 in case of F0^[<], we can 
convert a formula ip into an equivalent disjoint union of exponentially many unambiguous 
automata of polynomial size (in \ip\ and |S|) and the RMC. Using polynomial space we can 
generate each automaton, calculate the probability that the RMC generates an accepting 
trajectory by Proposition |7.2| , and sum these probabilities for each automaton. O 

Corollary 7.6. The probability of a TL[0, <$>] or F0^[<] formula holding on a hierarchical 
Markov chain can be computed in PSPACE. 

Proposition 7.7. The probability of an FO^ formula holding on an RMC can be computed 
in EXPSPACE. 



Proof. The result follows by the same argument as in Proposition |7.5[ but now by Theorem 
|4.3| we have family of doubly exponentially many automata each of exponential size, with 
a non-deterministic expo nential time algorithm for building each automaton. Therefore 



applying Proposition 7.2 we immediately obtain upper bounds for FO'^. □ 



For an ordinary Markov chain, calculating the probability of an LTL formula can be 
done in PSPACE |YanlO| . while we have seen previously that we can calculate the prob- 
ability of an FO^ formula in PEXP. One can achieve the same bounds for LTL and FO^ 
on hierarchical Markov chains. In each case we expand the HMC into an ordinary Markov 
chain and then use the model checking algorithm for a Markov chain. This does not impact 
the complexity, since the space complexity is only polylog in the size of the machine for 
LTL and the time complexity is only polynomial in the machine size for FO^. We thus get: 

Proposition 7.8. The probability of a FO^ formula holding on a HMC can be computed in 
PEXP, while for an LTL formula it can be computed in PSPACE. 



7.3. Markov decision processes. Courcoubetis and Yannakakis |CY95j have shown that 
the maximal probability with which a scheduler can achieve an UTL objective on an MDP 
can be computed in 2EXPTIME. It follows from results of |ATM03j that even the qualitative 
problem of determining whether every scheduler achieves probability 1 is 2EXPTIME-hard. 
Combining the 2EXPTIME upper bound with the exponential translation from FO^ to 
UTL |EVWn2j yields a 3EXPTIME bound for FO^. Below we see that using our FO^-to- 
automaton construction we are able to improve this bound to 2EXPTIME. 

We begin with universal formulation of qualitative model checking MDPs. To deal with 
MDP's, we will make use of determinism in the limit. 
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Proposition 7.9. Determining whether for all schedulers a F0'^[<] -formula tp holds almost 
surely on a Markov decision process M is co-NP-complete. 

Proof. The corresponding complement problem asks whether there exists a scheduler a such 
that the probability of -193 is greater than 0. For this problem, there is an NP algorithm, 
as we now explain. In Courcoubetis and Yannakakis |CY95j . there is a polynomial time 
algorithm for qualitative model checking deterministic Biichi automata on MDPs. As noted 
there, the algorithm applies to automata that are deterministic in the limit as well. There- 
fore we can just guess a particular automaton Ai from the family of automata corresponding 



to -i</7, as described in Theorem 4.6 The theorem guarantees that this automaton will be 
deterministic in the limit. 

It is easy to see that the co-NP is tight, even for TL[0,<$>]j since qualitative model 
checking for MDPs generalises validity for both TL[0, ^] formulas, which is co-NP hard. D 

Proposition 7.10. Determining whether for all schedulers a UTL-formula (p holds almost 
surely on a Markov decision process Ai is in EXPTIME. For FO^ the problem IS m co- 
N EXP TIME. 



Proof. The result for FO^ follows along the lines of the proof of Proposition 7.9, but now 
we guess an automaton Ai of exponential size (using Theorem 4.3). 

Similarly, for UTL we can use Theorem |4.1[ We still have exponential sized automata 
Aj, but only exponentially many of them, so we can iterate over all of them, which gives us 
a single exponential algorithm. □ 

Note that here the FO^ problem is easier than the corresponding LTL problem, which 
is known to be 2EXPTIME-complete. 

For the existential case of the qualitative model-checking problem, an upper bound of 
2EXPTIME for all of our languages will follow from the quantitative case below. On the 
other hand the arguments from |ATM03j can be adapted to get a 2EXPTIME lower bound 



(see Proposition 7.18) even for qualitative model-checking TL[0, ^] in the existential case. 
Hence we have: 

Proposition 7.11. Determining if there is a scheduler that enforces a formula with prob- 
ability one is 2EXP TIME- complete for each o/TL[0,^], UTL, LTL, ¥0'^[<\ and FO^. 

We now turn to the quantitative case. We apply the translation from FO^ to determin- 
istic parity automata from Subsection |4.3[ along with the result that the value of a Markov 
decision process with parity winning objective can be computed in polynomial time |CH12] . 



Using Theorem 4.13 we immediately get bounds for FO'^ that match the known bounds for 
LTL: 

Proposition 7.12. We can compute the maximum probability of an FO^ formula if over 
all schedulers on a Markov decision processes A4 in 2EXPTIME. 



7.4. Stochastic two-player games with FO^ winning condition. We can reduce the 
qualitative case of stochastic two-player games to the case of ordinary two-player games 
using the following result of Chatterjee, Jurdzinski and Henzinger: 

Proposition 7.13 ( jCJH03) ). Every (universal) qualitative simple stochastic parity game 
with n vertices, m edges and d priorities can be translated to a simple parity game with 
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the same set of priorities, with 0(dn) vertices and 0(d{m + n)) edges, and hence it can he 
solved in time 0(d{m + n){nd)'^/'^ ). 

Now combining the reduction with our results for two-player games, we ascertain the 
complexity of stochastic two-player games: 

Corollary 7.14. The universal qualitative model checking problem for Stochastic two-player 
games (2^-player game) with FO^ winning condition is 2EXPTIME-complete. 

Proof. Hardness follows from 2EXPTIME-hardness for two-player games with FO^ winning 
conditions. Membership is a consequence of the above reduction and our bounds for two- 
player games (see Proposition 6.4 and Proposition 7.13). □ 



7.5. Lower bounds. We can get corresponding tight lower bounds for most of the proba- 
bilistic model checking problems. 

Proposition 7.15. The quantitative model checking problem for a TL[0, ^] formula i{j on 
a Markov chain Ai is #P-hard. 

Proof. The proof is by reduction from ^SAT. Let 99 be a prepositional formula over literals 
ai,a2, ■ . .an. We construct a Markov chain Ai such that each trajectory generated by 
M. corresponds to an assignment of truth values to literals ai, . . . a„, with each of the 2"- 
possible truth assignments arising with equal probability. We also construct a TL 
formula ip such that only trajectories of M that encode satisfying valuations contribute to 
the probability Pm{L{iP)). Therefore the number of satisfying assignments of the original 
propositional formula (/? is 2"Pa/((L(-0)). 

See Figure[4]for a depiction of the Markov chain Ai in case n = 3. All probabilities equal 
1/2, except those on transitions leading to the final vertex /. A path going through vertex 
Ui corresponds to assigning true to the literal aj and a path through a'^ to an assignment of 
false. We construct the TL[0,0] formula ip corresponding to the propositional formula 93 
by replacing each positive literal in 99 with and each negative literal -iCj in ip with 
<><■ 




Figure 4: Markov chain Ai for n = 3 



Recalling the upper bound from Proposition 7.3 we conclude that the quantitative model 
checking problem for TL[^, on Markov chains is #P-complete. D 
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Proposition 7.16. The quantitative model checking problem for FO^ on Markov chains is 
PEXP-hard. 

Proof. PEXP-hardness is by reduction from ttie problem of wlietlier a strict majority of 
computation paths of a given non-deterministic EXPTIME Turing macliine T on a given 
input / are accepting. The Markov chain generates a uniform distribution over strings of 
the appropriate length, and the formula checks whether a given string encodes an accepting 
computation of Ai . The ability of FO^ to check validity of such a string has already been 
exploited in the NEXPTIME-hardness proof for FO^ satisfiability in [ EVW02j . The details 
of this approach can be found in the proof of Proposition 9.3 



Combining with the upper bound from Proposition 7.4, the quantitative model checking 
problem for FO^ on Markov chains is PEXP-complete. D 

Turning to lower bounds for MDPs, note that co-NEXPTIME-hardness for FO^ is 
inherited from the lower bound for Markov chains. On the other hand, we can show that 
the EXPTIME bound for UTL is tight: 

Proposition 7.17. Determining whether for all schedulers a UTL-formula ip holds almost 
surely on a Markov decision process Ai is EXPTIME-hard. 

Proof. The argument is based on the idea of Courcoubetis and Yannakakis for lower bounds 
in the LTL case. We reduce the acceptance problem for an alternating PSPACE Turing 
machine to the problem of whether there is a scheduler that enforces that a UTL formula 
(p holds with positive probability. Thus we reduce to the complement of the problem of 
interest. 

Consider an alternating PSPACE Turing machine T with input I. Without loss of 
generality we assume that each configuration of T has exactly two successors and that T 
uses space at most n on an input / of length n. Then we can encode a branch of the 
computation tree of T as a finite string in which each configuration is represented by a 
consecutive block of n-|- 1 letters: one bit to represent the choice to branch left or right, and 
n letters to represent the configuration. Let .^^(7) be the language of infinite strings, each 
of which is an infinite concatenation of finite strings that encode accepting computations. 
It is standard that one can write a UTL formula ip that captures L^^i)- 

Next we describe the MDP M. Intuitively the goal of the scheduler is to choose a 
path through Ai so as to generate a word in -L'7i(/) . A high-level depiction of Ai is given in 
Figure [5j The boxes init-conf and next-conf contain gadgets that are used by the scheduler 
to generate the initial configuration and all successive configurations of T as strings of 
length n. The number of such strings is exponential in n, but clearly the gadgets can be 
constructed using only linearly many states. After producing an existential configuration of 
the Turing Machine, the scheduler sends control to the state sch, where it decides whether T 
should branch left or right. After generating a universal configuration, an honest scheduler 
sends control to pro, the only randomising state in Ai, where the branching direction T is 
selected uniformly at random. When the scheduler has successfully generated an accepting 
computation it visits acc, which is the only accepting state of Ai, and the simulation starts 
over again from the beginning. Only those computations that visit acc infinitely often and 
in which the scheduler behaves honestly satisfy ip. 

We claim that there exists a scheduler such that Pm{L{(p)) > if and only if T accepts 
its input. 
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If the Turing Machine T accepts its input, then the scheduler can simply follow the 
strategy from the alternating computation of T. Regardless of the choice made by the 
probabilistic opponent, the scheduler can always go to an accepting vertex with probability 
1. Therefore even if we repeat the whole simulation, for this scheduler Pa,/(L((^)) = 1, which 
is greater than as required. 

The infinite repetition is important in the second case, when the Turing Machine T 
rejects its input. If the process ran only once, it could happen that in the probabilistic 
choice, only one option would lead to a rejecting state, but it would not be chosen if the 
probabilistic opponent of the scheduler were unlucky. Therefore we repeat this process 
infinitely many times and thus guarantee that with probability 1 we will reach the rejecting 
vertex and then stay there forever, i.e. PM{L{ip)) will be as required. 

Combining with the upper bound from Proposition 7.10[ determining whether for all 



schedulers a UTL-formula holds with probability one on a Markov decision process is 
EXPTIME-complete. □ 

init conf 



re] 



pro 



sch 



1/2. 




next conf 



Figure 5: Sketch of the Markov decision process 



The above was a lower bound for checking whether all schedulers enforce the property 
with probability 1. We now show a tight lower-bound for the existence of a probability one 
scheduler: 

Proposition 7.18. Given a Markov decision process and a TL[0, formula, determining 
whether the formula holds with probability one for some scheduler is 2EXPTIME-hard. 

Proof. The proof is an adaptation of the 2EXPTIME-hardness proof of Alur et. al. for 
model checking TL [<>,<$>] formulas on two-player games in [ ATM03J . The proof there is 
based on a reduction from the membership problem for an alternating exponential-space 
Turing machine, where a game graph and a TL[0>, formula are constructed such that the 
Turing machine accepts the given input if and only if the existential player has a winning 
strategy in the game. 

We can adapt the proof by assigning the existential vertices of the game graph to a 
scheduler and assigning the universal vertices from the game graph to the probabilistic 
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player (by setting the uniform outgoing probabilities from these vertices). When the Tur- 
ing machine accepts its input we are guaranteed that there is a corresponding scheduler 
that leads to acceptance with the probability 1. On the other hand, if the Turing machine 
does not accept its input then after some finite number of transitions in the Markov deci- 
sion process, either the scheduler "cheats" (does not follow the Turing machine transition 
function or cell numbering) or we get to a rejecting state. In both cases, the probability of 
acceptance is less than 1. □ 



Table 7.5 summarises the known results and the results from this paper (in bold) on 
probabilistic systems. An asterisk indicates bounds that are not known to be tight. Note 
that for the more complex verification problems, from strategy synthesis for MDPs onwards, 
all problems are 2EXP-complete. Intuitively the complexity of the model overwhelms the 
difference in the respective logics. Similarly, we see that in the stutter-free case the extra 
succinctness of F0^[<] comes at "no cost" over TL[O)0] — at least, for the complexity 
classes we consider, and where we can establish tight bounds, the respective columns are 
identical. 





TL[0,^] 


UTL 


F0^[<] 


FO^ 


LTL 


Markov chain 


#P 


PSPACE 


#P 


PEXP 


PSPACE 


HMC 


PSPACE* 


PSPACE 


PSPACE* 


PEXP 


PSPACE 


KMC 


PSPACE* 


EXPSPACE* 


PSPACE* 


EXPSPACE* 


EXPSPACE* 


MDP (V) 


co-NP 


EXP 


co-NP 


co-NEXP 


2EXP 


MDP (3) 


2EXP 


2EXP 


2EXP 


2EXP 


2EXP 


MDP (quant) 


2EXP 


2EXP 


2EXP 


2EXP 


2EXP 


2i-game (V) 


2EXP 


2EXP 


2EXP 


2EXP 


2EXP 



8. Model checking F02[LTL] 

We now turn to combining FO^ with automata-based techniques for LTL, examining verifi- 
cation of the hybrid language FO^ [LTL] . As was done with FO^ , we first show that we can 
translate FO^[LTL] into temporal logic with exponential blow-up in the size of the formula, 
giving a simple upper bound. While for FO^ the translation was to unary temporal logic, 
in this case we have a translation to LTLLgf 

We can look at every FO^[LTL] formula as being rewritable using let definitions such 
that every let definition involves either a pure FO^ formula or a pure LTL operator. We get 
this form by introducing a let definition for every subformula with one free variable. For 
example, rewriting the formula ip = {{3y {suc{x , y) APi(x))) U Pq){x) with let definitions 
yields 

(fiet = Let Ro{x) be Po{x) in 
Let Ri{x) be Pilx) in 
Let R2{x) be 3y {suc{x, y) A Ri{x)) in 
{R2URq){x) 

Note that although the above uses a combination of FO^ and LTL, each individual 
definition is either "pure FO^" , or "pure LTL" , and we can apply the translation of FO^ to 



UTL in Lemma 3.10 to each FO definition. This gives the following result: 
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Lemma 8.1. Given an FO^[LTL] formula ip, we can convert it to an equivalent LTLlgi 
formula tp such that {ipl = 0(21^1''). 

We could then translate the let definitions away for LTL, to get an ordinary LTL 
formula — thus showing that FO^[LTL] and LTL have the same expressiveness. However, 
there is no need to perform this second transformation to get a bound on the complexity of 
model checking. Let definitions do not increase complexity for model checking LTL, since 
non-deterministic Biichi automata for LTL and LTLL^t have the same asymptotic size: 

Lemma 8.2. Given an LTLlci formula (p, there is an unambiguous Biichi automaton A 
with at most 0(2l'^l ) states accepting exactly the language {w G : w \= tp}. Moreover 
this automaton can be constructed in polynomial time in its size. 

This follows from the fact that the number of subformulas of LTLLet formulas is linear 



in the formula size (Lemma 2.1) and from the following result of Couvreur et al: 

Lemma 8.3 ( |CSS03j ). Given an LTL formula ip, there is an unambiguous Biichi automa- 
ton A with at most 0(|S||sub(99)|2l^"'^('^l)) states accepting exactly the language {w : w G 
T,^ Aw 1= v?}. Moreover this automaton can be constructed in polynomial time in its size. 

As a corollary of Lemmas 8.1 and 8.2 we see that we can convert from an FO^[LTL] 



formula to an unambiguous Biichi automaton in doubly exponential time, giving a doubly- 
exponential bound on the complexity of model-checking. However, just as in the previous 
section, we show that we can do better by direct analysis than via this translation approach. 



We begin by looking at the translation given in Lemma 8.1 from a different perspective. 
Let us extend the set of atomic propositions V and alphabet S = 2^ by adding new atomic 
propositions TZ for every predicate created in that translation. Thus we have an extended 
alphabet S' = 2^^^. There is an obvious restriction mapping taking an infinite word w' 
over S' to a word over S, simply by ignoring all propositions in TZ; we denote this by 
restrict (if', S). 

Lemma 8.4. Given an FO^[LTL] formula ip alphabet S, there is an FO^ formula ipp and 
an LTL formula ip^ over T,' having the following two properties for all w £ Ti^ : (i) if w |= ip 
then there is a unique extension w' of w such that w' \= ipi A ipFj (H) if w ^ ip then there 
is no extension to w' such that w' |= tpL A ipp- Moreover, \<pl\, \ fF\ = 0(|93p) 



Proof. We use the translation in Lemma 8.1, but consider it simply returning the collection 
of let definitions. Corresponding to each definition is a conjunct stating that Ri holds iff ipi 
holds. We now examine the form of this conjunct. 

Each ipi is either a basic two-variable formula or an LTL atomic formula. If ipi is in 
LTL then the iff can be expressed again in LTL: O ipi). If ipi is in FO^ then the iff 

above can be expressed as \/x.{Ri{x) -H- ipi{x)). We can simply let 93^? be th FO^ conjuncts 
and (pL i>G the LTL conjuncts to obtain the desired conclusion. 

The upper bounds for lengths \ipL\ and \ipF\ follow from the fact that k < \<p\ and 

\^i\ < I'/'l- □ 

For the formula from the example at the beginning of this section we get following 
formulas ipi and ipp over S': 

ipL = iR2U Ro){x) An{Ro{x) ^ Poix)) A 

\JiRi{x) o Pi(x)) 
ipF = Vx.(i?2(a^) -H- 3y.(suc(x, y) A 
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8.1. Combining automata constructions for FO and LTL. Given FO [LTL] formula 



(f, we can apply Lemma 8.4 to obtain an equisatisfiable formula (fi A (fp, where ipi is an 
LTL formula and ifp is an FO^ formula over the extended alphabet S'. Now we can build a 

emma 18.31 as well collection 



4.3 



Biichi automaton Bi for ^pi using the construction from 
of 2^''°'''"'^^" Biichi automata Bp^ for ipp, using Theorem 

For each i we build a product automaton Ai = Bl ® Bp. synchronising on the truth 
values of the newly introduced atomic propositions Ri. We claim that each product automa- 
ton Ai is unambiguous, the languages they accept are disjoint, and their union is exactly 
{w £ T,^ : w 1= v?}. This follows from the fact that each word over E has only one extension 
to a word over T,' for which B^ accepts, along with the fact that the languages accepted by 
the Bp- are disjoint. 

After producing the synchronised cross product, we can restrict the input alphabet back 
to S, because the values of all newly introduced atomic propositions G S' \ S are fully 
determined by the truth values of atomic predicates Pi and the relations defined by ^p. 

Therefore we get the following theorem: 

Theorem 8.5. FO^[LTL] formula p, there is a collection of doubly exponentially many (in 
\(p\) generalized Biichi automata Ai, each of exponential size in \ip\, such that the languages 
they accept are disjoint and the union of these languages is exactly {w G Ti^ : w |= (p}. 
Moreover, each automaton Ai is unambiguous and can be constructed by a non- deterministic 
Turing machine in polynomial time in its size. 

This translation will now allow us to read off bounds for many FO^[LTL] verification 
problems. 



we can 



8.2. Model Checking FO^[LTL]. Comparing Theorem 4.3 with Theorem 8.5 
easily see that automata for FO^ in isolation and FO^[LTL] have the same asymptotic size. 
We can therefore use all automata-based bounds on verification results for FO^, provided 
that they rely only on unambiguity of the resulting automata. This allows us to replace 
FO^ with F02[LTL] in the results of the previous sections, giving the following: 

Proposition 8.6. Model checking FO^[LTL] properties on Kripke structures, hierarchical 
and recursive state machines is in the complexity class NEXP. 

Proposition 8.7. The threshold problem for model checking FO^[LTL] on both Markov 
chains and hierarchical Markov chains is in PEXP. 

Proposition 8.8. The probability of an FO^[LTL] formula holding on a recursive Markov 
chain can be computed in EXPSPACE. 

Now let us consider model checking Mark ov dec ision processes. Recall that in the proof 



of the corresponding bound for FO^, Theorem 7.10, we relied on the fact that the automata 



are deterministic in the limit. Thus our translation for FO^[LTL] does not give us the 
same bounds as for FO^. And indeed, the corresponding bound for checking whether all 
schedulers achieve probability 1 is worse for LTL in this case, namely doubly-exponential. 
We will show that we can achieve the same bound as for LTL. 

Proposition 8.9. Determining whether for all schedulers an FO^ [LTL] -/ormw/a ip holds on 
a Markov decision process with probability one is in the complexity class 2EXPTIME. 
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Proof. We will decide the corresponding complement problem which asks whether there 
exists a scheduler a such that the probability satisfying -199 is greater than 0. By applying the 
translation from Theorem 8.5, we get a collection of doubly-exponentially many automata, 
each of exponential size. We can go through all these automata and check if the probability 
is greater than for one of them. For each automaton, we make a call to the exponential 
time algorithm for qualitative model checking Biichi automata on MDPs from Courcoubetis 
and Yannakakis jCY95] . D 

The following table summarises the results for FO^[LTL] from this paper (in bold) 
concerning both non-deterministic and probabilistic systems in the context of results for 
FO^ and LTL alone. An asterisk indicates bounds that are not known to be tight. The table 
shows that for the models considered in this paper the complexity of verifying FO^[LTL] is 
the maximum of the respective complexities of FO^ and LTL. 





FO^[LTL] 


FO^ 


LTL 


Kripke structure 

HSM 

RSM 


NEXP 
NEXP 
NEXP 


NEXP 
NEXP 
NEXP 


PSPACE 
PSPACE 
EXPTIME 


Markov chain 
HMC 
RMC 
MDP (V) 


PEXP 
PEXP 
EXPSPACE* 
2EXP 


PEXP 
PEXP 
EXPSPACE* 
co-NEXP 


PSPACE 
PSPACE 
EXPSPACE* 
2EXP 



9. The impact of Let definitions on model checking 

In the process of examining two- variable logics and their extensions, we have utilized re- 
sults on logics extended with Let definitions. We now return to considering the impact of 
Let for several temporal logics. First, we note that model checking TL[0,^]Lct, UTLlci 
and LTLLet properties on both non-deterministic (Kripke structures, HSMs, RSMs) and 
probabilistic systems (Markov chains, HMCs, RMCs, MDPs (V)) has similar computational 
complexity as for the corresponding logics without let definitions. We get these results by 
simply substituting let definitions to obtain formulas in the base logic, and then analyze 
the complexity of model-checking the resulting formulas. 

In the case of LTLlcI! we have already noted that the size of the automaton for LTL is 
exponential only in the number of subformulas (see, e.g. Couvreur et. al. |CSS03| ) — this 



leads to Lemma 8.2 Similarly, for TL[0,<$>]Let and UTLLet, we get the corresponding 
automata of the same asymptotic size as for TL[0, <3>] and UTL respectively, because their 
size depends on the number of subformulas and the operator depth and not directly on the 
size of the formula (see translation in Subsect ion |4.1[ ). 

In the case of FO^gt, we can use Lemma 3.10 to translate the formula to UTLLet and 



then use the result above that the sizes of the automata for UTL and UTLLet formulas of the 
same length are asymptotically equal. Moreover, since LTLLet and FOLg^ have unambiguous 
Biichi automata of equal asymptotic size as for LTL and FO^ respectively, we can combine 
them in the same way as in the proof of Theorem |8.5| to get the same complexity upper 
bounds for model checking FO^ [LTL] Let as for FO^[LTL]. Thus we have: 
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Proposition 9.1. For LTL, FO^, UTL, TL[0,^], and FO^[LTL], all the upper bounds 
previously shown hold also in the presence of Let definitions. 

Finally, we will show that, in contrast to the cases above, the complexity of model check- 
ing FO^[<]Let is exponentially worse than that of F0^[<] on both non-deterministic and 
probabilistic systems. Thus this is the only logic we have considered where the introduction 
of let definitions makes a difference in the computational complexity of model checking. 
The following two theorems show the lower bounds on the complexity of model checking 
FO^[<]Let) which match exactly the upper bounds for FO^p^ (compare with Proposition 



6.2) 



Proposition 9.2. The satisfiability of a FO^[<]Lct formula under the unary alphabet re- 
striction is NEXP-hard. 

Proof. The proof is by reduction from the halting problem of a non-deterministic EXPTIME 
Turing machine T on a given input /. Let F and Q be respectively the tape alphabet and 
set of control states of T. We consider infinite strings over alphabet 

S := ({Po, Pi...- P2n-i] X {F U (F X Q)}) U {#} . 

An infinite word u G S"^ encodes a computation of T as follows. Each configuration is 
encoded in a block of contiguous letters in u, with successive configurations arranged in 
successive blocks. Each such block comprises 2" symbols denoting the contents of each tape 
cell in the configuration. A symbol encoding a tape cell consists of: a letter from FU (F x Q) 
to denote the contents of the tape cell and whether the read head of the Turing Machine is 
currently on the cell (and if so, the current control state of T), and a predicate Pi denoting 
the address of the tape cell and the configuration number. Here we use the power of Let 
definitions to transform the sequence of 2n predicates to values of 2n-bit counter (see the 



proof of Lemma 3.12), which represent the address of configuration and tape cell. Having 
thus encoded a computation of T in a finite prefix of u we require that the remaining infinite 
tail of u be the string 7^'^. 

We can use short FO^[<]Let formulas to identify the position in the string representing 
the previous or next position of the tape cell in the same configuration. We can also use such 
formulas to identify the same position of the tape cell in the previous or next configuration. 
Thus we can easily check if the tape symbols are consistent with the transition function 
of T. Finally, we ensure T is in the accepting state in the last configuration. O 

Proposition 9.3. The decision problem of whether a Markov chain M. satisfies an FO^[<]Let- 
formula ip with probability greater than 1/2 is PEXP-hard. 

Proof. The proof is by reduction from the problem of whether a strict majority of compu- 
tation paths of a given non-deterministic EXPTIME Turing machine T on a given input / 
are accepting. Without loss of generality we can assume that any non-halting configuration 
of T has exactly two successors and that all computations of T on input / make exactly 2"' 
steps, where re is the length of /. 

The basic idea, following the proof of NEXPTIME- hardness of satisfiability for FO^ [<] Let , 
is to encode computations of T as strings. We can define an FO^[<]Lct formula that is sat- 
isfied by a word u G precisely when u encodes a legitimate computation of T on input 



/ according to the encoding scheme used in Proposition 9.2 Indeed, the definition is just 
as described in the proof of NEXPTIME-hardness for FO'^[<]Let satisfiability in Proposi- 
tion [9l2 
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The Markov Chain M. in our reduction is constructed from two copies of a component 
M'. The definition of M' is very simple; it consists of a directed chque augmented with a 

single sink state. In detail, there is a state for each letter o" G E; is a sink that makes 
a transition to itself with probability 1; the next-state distribution from s^; <7 7^ #, is given 
by a uniform distribution over all states; finally, the label of state is a. 

The Markov chain M consists of two disjoint copies Mieft and Mright of that are 
identical except that their states arc distinguished by propositions Pieft and Pright- 

The 

initial state of is a uniform distribution over all states. 

We can partition S'^ into three sets A and R, respectively comprising those strings 
that don't encode computations of T on input /, those strings that encode accepting com- 
putations, and those strings that encode rejecting computations. Moreover each of these 
sets is definable in FO^[<]Let by formulas (p^, ipA and (pR respectively. 

We define the formula (p by 

<p := {{MxPieftix)) A ((^7v V 'Pa)) V {{MxPrightix)) A pa) ■ 
To complete the reduction, we claim that Pm{L{p)) > 1/2 if and only if a strict 
majority of the computations of Turing Machine T on input / are accepting. To see this, 
observe that if Ai produces a trajectory from N C T,^ then that trajectory is equally likely 
to have come from Aiieft or Mright- Using this we can see that Pm{L{p)) is {Pm{A) + 
Pm{N))/2 + Pm{A)/2. Thus Pm(L((^)) > 1/2 iff 2Pm{A) > 1 - Pm{N). Prom this we 
see that Pa4(L(<^)) > 1/2 if and only if \A\ > \R\, as required. □ 

The table below summarises the results for the selected logics. An asterisk indicates 
bounds that are not known to be tight. 





TL[0,^]Let 


FO^[<]Let 


FOLt 


FO^[LTL]Let 


Kripke structure 


NP 


NEXP 


NEXP 


NEXP 


HSM 


NP 


NEXP 


NEXP 


NEXP 


RSM 


NP 


NEXP 


NEXP 


NEXP 


Markov chain 


#P 


PEXP 


PEXP 


PEXP 


HMC 


PSPACE* 


PEXP 


PEXP 


PEXP 


RMC 


PSPACE* 


EXPSPACE* 


EXPSPACE* 


EXPSPACE* 


MDP (V) 


co-NP 


co-NEXP 


co-NEXP 


2EXP 



10. CONCLUSIONS AND ONGOING WORK 

In this paper we have compared the complexity of verifying properties in the two best- 
known elementary fragments of monadic first-order logic on words: LTL and FO^. We 
provided several different logic-to-automaton constructions that are useful for verification 
of FO^. One translations allows us to understand the complexity of verifying full FO^ 
via analysis of unary temporal logic; a second is useful for the sublanguage of FO^ with 
only the linear-ordering; the third is useful for getting deterministic automata, which is 
needed for obtaining bounds for certain game-related problems. We have shown that these 
translations put together allow us to understand the complexity of verification and synthesis 
problems for both non-deterministic and probabilistic models transition systems, including 
those arising from hierarchical and recursive state machines. 
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While LTL is more expressive than FO^, FO^ can be exponentially more succinct. We 
have shown that the effect of these opposing factors on the complexity of model checking de- 
pends on the model, e.g., FO^ has higher complexity on Markov chains while LTL has higher 
complexity on MDPs. By contrast, in the stutter- free case the extra succinctness of F0^[<] 
comes for free — all verification problems have the same complexity as for TL[0,^]- For 
the most structured models e.g., two- player games and quantitative verification of MDPs, 
the complexity of the model dominates any difference in the logics. 

We are currently examining the succinctness of Let definitions when added to each of 
our logics. A number of succinctness results can be found in this work, but we have left 
open the succinctness of Let in certain situations, e.g., for the logic FO^[LTL]. Finally, we 
are investigating the extension of the techniques introduced here from words to trees. 
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